EzDevInfo.com

SOAP vs REST (differences)

I have read articles about the differences between SOAP and REST as a web service communication protocol, but I think that the biggest advantages for REST over SOAP are:

1. REST is more dynamic, no need for creating and updating UDDI.

2. REST is not restricted to XML format. REST web services can send plain text, JSON, and also XML.

But SOAP is more standardized (Ex; security).

So, am I correct in these points?

REST / SOAP endpoints for a WCF service

I have a WCF service and I want to expose it as both a RESTfull service and as a SOAP service. Anyone has done something like this before?

Is an entity body allowed for an HTTP DELETE request?

When issuing an HTTP DELETE request, the request URI should completely identify the resource to delete. However, is it allowable to add extra meta-data as part of the entity body of the request?

How to do authentication with a REST API right? (Browser + Native clients) [closed]

I'm building a web application using Rails. At the moment I'm using Devise with HTTP sessions which was pretty easy to set up and it's working well.

The application consists of one URL providing an AJAX web application. The rest of the URLs available belong to the REST API. So everything and every little data request is done via AJAX.

Now I'd like to extend the whole thing to support native clients. I read a lot about stateless auth, http basic and digest auth, http sessions, cookies, xsrf, etc... And now I feel like I can't have a secure app, because there's always a way to hijack some parts of it.

1.: HTTP session Vs. stateless auth token

What's the difference? I don't get it.

• HTTP session:

  1. Client requests a URL (first request to the server)
  2. Server gives the normal response plus some unique string (== session ID)
  3. Client has to send this string with every request (which is done automatically using HTTP header)
  4. Client logs in -> Server memorizes that this particular session ID is now logged in
  5. Client visits a page which requires auth -> Nothing special to do, because the session ID will automatically get sent to the server via HTTP header

• stateless auth token:

  1. Client request URL (first request to the server)
  2. Server just gives the normal response without any key or token or id
  3. (nothing special here)
  4. Client logs in -> Server creates an auth token and sends this token to the client inside the response
  5. Client visits page which requires auth -> Client has to submit the auth token

For me both ways look pretty similar. With Rails I can also choose to store the session inside the database... Devise would do the same with the stateless auth token.

2.: The authentication method

Right now I'm using POST /users/sign_in with {"user":{"email":"e@mail.com","password":"p455w0rd"}}.

But there are other possibilities like HTTP basic auth and HTTP digest auth, but also solutions like oAuth (too big for my purpose).

From what I've read:

• Concerning sign_in security there's no difference between the current POST /users/sign_in and HTTP basic auth. Both use cleartext.
• For sign_out HTTP basic auth has a disadvantage: Sign out is only possible closing the browser window
• HTTP digest auth has a huge advantage: It doesn't transmit the password at all (just a hash of password plus random generated string)
• (German) Wikipedia says: HTTP digest auth is not supported by all browsers. Maybe this information is way to old?!

What I need:

• usernames and hashed passwords (bcrypt) stored in a database.
• user can change his password and the password has not to be sent in plaintext. (The same problem occurs when it comes to user sign_up). Possible solutions?
  1. of course: using SSL/TLS
  2. client request a want_to_change_password_salt and uses it to encrypt the password on client side. but (?!) this way I'd sent an essential part of the hashed password over the wire plus the hashed password. Sounds insecure to me?!

3.: CSRF Token

As said above, right now I have just a normal AJAX website using the REST API. It has XSRF protection: The website gets delivered by rails and thus has embedded the XSRF token. I read it using AJAX and transmit it when doing a POST. Rails then returns the requested data and a new XSRF token, which I then use for the next POST.

Now I want to change my server application to work with native clients. A native client won't load the HTML page and thus won't retrieve a CSRF token. So the following options came to my mind:

• Create a XSRF token REST resource. So the (native) client has to request a XSRF token from this resource before it can do the first POST.
• Disable XSRF protection entirely.

Questions:

• How does XSRF protection work (in Rails)? How does the server know which token belongs to which client? The only way I can think of are sessions. This assumption leads to:
• If I disable session in order to create a fully stateless REST API, XSRF protection won't work anymore. Right?

4.: Stateless auth token

Here I have mostly a lot of questions:

• Does it have the same security problems as HTTP sessions? What I mean: Stealing the session ID has the same effect as stealing the auth token. Right?
• Expiration of the auth token should work the same as with HTTP sessions: The server has to store somewhere (database respectively session) a timestamp and check that.
• sign_out works the same, too?
  • Session: Destroy session on the server
  • Auth token: Destroy the token on the server
• From what I've read it should be more secure to store the auth token inside the HTTP header (just like session ID), because server logs can contain GET parameters and thus could contain the token.
• Should it just be a plain auth token or would it be better if the client also transmits its user_id or even the hashed password? HERE I read that the client should send:
  1. user_id
  2. expiration_date
  3. a hash (or what's HMAC?) of [user_id, expiration_date, SECRET_KEY]. Where SECRET_KEY is basically a random string generated by the server.

Sorry for the huuuge post, but security is essential! And I don't want to make design mistakes which could probably expose private data.

Thank you :)

Here's a bit of new information and new questions ;-):

5.: Native Clients

As far as native clients are concerned, there's no (easy) way to use sessions:

• A native client is no browser

• Thus it won't easily handle cookies (and without cookies there's no typical session handling)

So there are 3 possible choices:

1. Implement session handling for native clients. This would be like:

   1. Login
   2. read HTTP Header of response to get the cookies
   3. save all cookie data you need (especially the one with the session stuff) locally
   4. send this session id with every request you do

2. Don't use sessions at all. From the point of view of a native client it's pretty much the same as 1.:

   1. Login
   2. Get some authentication token from HTTP Header or response body (it's your app, though it's up to you)
   3. save this token locally
   4. send this token with every request

3. The hybrid approach. This basically means, that the server has to distinguish between browser and native client and then check the provided session id and session data or (for native clients) check the provided auth token.

6.: CSRF Token with stateless (= no session/no cookies) auth

CSRF Protection protects your users from malicious websites, that try to do some request on your API in the name of your logged in user, but without your user knowing it. That's pretty simple when using sessions:

1. User logs in at your API
2. Session get's created
3. Your users browser will have a cookie set with this session ID
4. Every request your user does do your API is automatically authenticated, because the browser will send all cookies (including the session id) along with each request to your API

And thus the attacking website simply has to do the following:

1. Write a custom HTML <form> which points to your API
2. Let the user somehow click the Submit button

Of course this form will be something like:

<form action="http://your.api.com/transferMoney" method="post">
  <input type="hidden" name="receiver" value="ownerOfTheEvilSite" />
  <input type="hidden" name="amount" value="1000.00" />
  <input type="submit" value="WIN MONEY!!" />
</form>

This leads to the following assumptions:

1. CSRF Protection is only needed because browsers automatically send cookies.

2. Native clients to not need CSRF Protection (of course: your browser can't access the authentication data (token, cookie, whatever) of your native app, and your native app won't use a browser to communicate with the API)

3. If you've got an API design which doesn't use Cookies to authenticate the user, there's no possibility to do CSRF. Because the attacker must know the authentication token and explicitly send it along with the malicious request.

If you want to oversecure your app, you can of course use CSRF Tokens along with you stateless authentication mechanism, but I'm pretty sure, that there's no additional security gain.

7.: The right HTTP Methods to choose

Login / Sign in and Logout / Sign out:

Never use GET for (at least) three reason:

1. CSRF Protection in most cases only protects POST, PUT, PATCH and DELETE and thus a CSRF could login a user without his knowledge when using a GET request

2. GET requests should never change the application state. But when i.e. using Sessions the application state changes on login/logout, because a session gets created or destroyed.

3. When using a GET request and transmitting the authentication information as URL parameters (i.e. http://your.api.com/login?username=foo&password=bar) there is another problem: Server logs! Most servers simply log every HTTP request including all URL parameters. That means: If your server get's hacked, there's no need to crack the password hashes from your DB, they must just have a look at the server's log files. In addition a malicious admin could also read the login information for every user. Solutions:

   • Use POST (or whatever method you like) and send the authentication information inside the request body. Or:
   • Send the authentication information within the HTTP headers. Because those information normally do not appear in the server log files. Or:
   • Have a look at the server config, and tell it to remove every URL parameter that is named "password" (or obfuscate is, so the URL becomes login?username=foo&password=*** inside the logs). But I suggest, to simply use the request body for this kind of information along with the POST method.

So you could use for example:

POST http://your.api.com/authentication for login

DELETE http://your.api.com/authentication for logout

8.: Passwords and Hashing

Authentication only works with some secret key. And of course this key should be kept secret. This means:

• Never store a password in plaintext in your database. There are several libraries available to make it secure. In my opinion the best option is bcrypt.

• bcrypt: It's been optimized to hash passwords. It automatically generates a salt and hashes the password multiple times (rounds). In addition the generated hash-string contains everything needed: Number of rounds, salt and hash. Though you just need to store this one String and there's no need to write anything by hand.

• of course you can also use any other strong hashing library. But for most of them, you've got to implement salting and using more than 1 rounds yourself. Additionally they wont give you just a single string like bcrypt does, though you've got to manage yourself to store rounds, salt and hash and reassemble it afterwards.

• rounds: This is simply how often the password should be hashed. When using 5000 rounds the hashing function will return the hash of the hash of the hash of the hash of the password. There's basically a single reason to do this: It costs CPU Power! This means: When someone tries to bruteforce your hash, it takes 5000 times longer when using 5000 rounds. For your application itself it doesn't matter that much: If the user knows his password, he will not recognize, if the server took 0.0004ms or 2ms to validate it.

• good passwords: The best hashing function is useless, if the password is too simple. If it can be cracked, using a dictionary, it doesn't really matter if you hashed it with 5000 rounds: It will maybe take a few hours longer, but what are a few hours, if it could be months or years? Though make sure, that your user's passwords contain the usual recommendations (lower + upper case + numbers + special chars, etc. pp.)

9.: Sending encrypted passwords over the wire

If you can't (or don't want to) rely on HTTPS, but don't want to send passwords in cleartext when signing in, you can use asymmetric cryptography ( http://en.wikipedia.org/wiki/Public-key_cryptography ).

This server creates a key pair (public key and private key). The public key is made available to the clients, the private key has to be kept private!

The client can now encrypt data using the public key, and this data can only be decrypted by the owner of the private key (= the server).

This should not(!) be used to store passwords in the database, because if your server gets hacked, the hacker will have the encrypted passwords and the private key for decryption. Though keep using some hashing algorithm (like bcrypt) for storing passwords in your database. Another reason is, that you can easily generate a new key pair, if you think that someone cracked you encryption.

HTTPS basically works the same way. Though, if your application uses HTTPS (which is recommended) there might be no big benefit in terms of security. But as stated above, if you can't use HTTPS for whatever reason or don't trust it, that's a way to craft your own secure connection.

And keep in mind that a real HTTPS connection encrypts the whole(!) connection and all data, not only password data. And it encrypts it both ways, from client to server and server to client.

What exactly is RESTful programming?

What exactly is RESTful programming?

SOAP or REST for Web Services?

Is REST a better approach to doing Web Services or is SOAP? Or are they different tools for different problems? Or is it a nuanced issue - that is, is one slightly better in certain arenas than another, etc?

Bounty-Edit:

Now, almost three years later I would like to ask this question again - offering a bounty to encourage an indepth answer. I would especially appreciate information about those concepts and their relation to the PHP-universe and also modern high-end web-applications.

PUT vs POST in REST

According to the HTTP/1.1 Spec:

The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line

In other words, POST is used to create.

The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI."

That is, PUT is used to create or update.

So, which one should be used to create a resource? Or one needs to support both?

Separate REST JSON API server and client?

I'm about to create a bunch of web apps from scratch. (See http://50pop.com/code for overview.) I'd like for them to be able to be accessed from many different clients: front-end websites, smartphone apps, backend webservices, etc. So I really want a JSON REST API for each one.

Also, I prefer working on the back-end, so I daydream of me keeping my focus purely on the API, and hiring someone else to make the front-end UI, whether a website, iPhone, Android, or other app.

Please help me decide which approach I should take:

TOGETHER IN RAILS

Make a very standard Rails web-app. In the controller, do the respond_with switch, to serve either JSON or HTML. The JSON response is then my API.

Pro: Lots of precedent. Great standards & many examples of doing things this way.

Con: Don't necessarily want API to be same as web app. Don't like if/then respond_with switch approach. Mixing two very different things (UI + API).

REST SERVER + JAVASCRIPT-HEAVY CLIENT

Make a JSON-only REST API server. Use Backbone or Ember.js for client-side JavaScript to access API directly, displaying templates in browser.

Pro: I love the separation of API & client. Smart people say this is the way to go. Great in theory. Seems cutting-edge and exciting.

Con: Not much precedent. Not many examples of this done well. Public examples (twitter.com) feel sluggish & are even switching away from this approach.

REST SERVER + SERVER-SIDE HTML CLIENT

Make a JSON-only REST API server. Make a basic HTML website client, that accesses the REST API only. Less client-side JavaScript.

Pro: I love the separation of API & client. But serving plain HTML5 is quite foolproof & not client-intensive.

Con: Not much precedent. Not many examples of this done well. Frameworks don't support this as well. Not sure how to approach it.

Especially looking for advice from experience, not just in-theory.

Pagination in a REST web application

This is a more generic reformulation of this question (with the elimination of the Rails specific parts)

I am not sure how to implement pagination on a resource in a RESTful web application. Assuming that I have a resource called products, which of the following do you think is the best approach, and why:

1. Using only query strings

eg. http://application/products?page=2&sort_by=date&sort_how=asc
The problem here is that I can't use full page caching and also the URL is not very clean and easy to remember.

2. Using pages as resources and query strings for sorting

eg. http://application/products/page/2?sort_by=date&sort_how=asc
In this case, the problem that is see is that http://application/products/pages/1 is not a unique resource since using sort_by=price can yield a totally different result and I still can't use page caching.

3. Using pages as resources and an URL segment for sorting

eg. http://application/products/by-date/page/2
I personally see no problem in using this method, but someone warned me that this is not a good way to go (he didn't give a reason, so if you know why it's not recommended, please let me know)

Any suggestions, opinions, critiques are more than welcome. Thanks.

Posting a File and Data to RESTful WebService as JSON

This is probably going to be a stupid question but I'm having one of those nights. In an application I am developing RESTful API and we want the client to send data as JSON. Part of this application requires the client to upload a file (usually an image) as well as information about the image.

I'm having a hard time tracking down how this happens in a single request. Is it possible to Base64 the file data into a JSON string? Am I going to need to perform 2 posts to the server? Should I not be using JSON for this?

As a side note, we're using Grails on the backend and these services are accessed by native mobile clients (iPhone, Android, etc), if any of that makes a difference.

If REST applications are supposed to be stateless, how do you manage sessions?

I'm in need of some clarification. I've been reading about REST, and building RESTful applications. According to wikipedia, REST itself is defined to be Representational State Transfer. I therefore don't understand all this stateless gobbledeygook that everyone keeps spewing.

From wikipedia:

At any particular time, a client can either be in transition between application states or "at rest". A client in a rest state is able to interact with its user, but creates no load and consumes no per-client storage on the set of servers or on the network.

Are they just saying don't use session/application level data store???

I get that one goal of REST is to make URI access consistent and available, for instance, instead of hiding paging requests inside posts, making the page number of a request a part of the GET URI. Makes sense to me. But it seems like it is just going overboard saying that no per client data (session data) should ever be stored server side.

What if I had a queue of messages, and my user wanted to read the messages, but as he read them, wanted to block certain senders messages coming through for the duration of his session? Wouldn't it make sense to store this in a place on the server side, and have the server only send messages (or message ID's) that were not blocked by the user?

Do I really have to send the entire list of message senders to block each time I request the new message list? The message list pertinent to me wouldn't/shouldn't even be a publicly available resource in the first place..

Again, just trying to understand this. Someone please clarify.

Update:

I have found a stack overflow question that has an answer that doesn't quite get me all the way there: How to manage state in REST which says that the client state that is important should all be transferred on every request.... Ugg.. seems like a lot of overhead... Is this right??

InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately [duplicate]

Tried to perform REST GET through python requests with the following code and I got error.

Code snip:

import requests
header = {'Authorization': 'Bearer...'}
url = az_base_url + az_subscription_id + '/resourcegroups/Default-Networking/resources?' + az_api_version
r = requests.get(url, headers=header)

Error:

/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:79: 
          InsecurePlatformWarning: A true SSLContext object is not available. 
          This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. 
          For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning

My python version is 2.7.3. I tried to install urllib3 and requests[security] as some other thread suggests, I still got the same error.

Wonder if anyone can provide some tips?

Thanks.

Understanding REST: Verbs, error codes, and authentication

I am (as a follow-up to this question) looking for a way to wrap APIs around default functions in my PHP-based web applications, databases and CMSs.

I have looked around and found several "skeleton" frameworks. In addition to the answers in my question, there is Tonic, a REST framework I like because it is very lightweight.

I like REST the best for its simplicity, and would like to create an API architecture based on it. I'm trying to get my head around the basic principles and have not fully understood it yet. Therefore, a number of questions.

1. Am I understanding it right?

Say I have a resource "users". I could set up a number of URIs like so:

/api/users     when called with GET, lists users
/api/users     when called with POST, creates user record
/api/users/1   when called with GET, shows user record
               when called with PUT, updates user record
               when called with DELETE, deletes user record

is this a correct representation of a RESTful architecture so far?

2. I need more verbs

Create, Update and Delete may be enough in theory, but in practice I will have the need for a lot more verbs. I realize these are things that could be embedded in an update request, but they are specific actions that can have specific return codes and I wouldn't want to throw them all into one action.

Some that come to mind in the user example are:

activate_login
deactivate_login
change_password
add_credit

how would I express actions such as those in a RESTful URL architecture?

My instinct would be to do a GET call to a URL like

/api/users/1/activate_login 

and expect a status code back.

That deviates from the idea of using HTTP verbs, though. What do you think?

3. How to return error messages and codes

A great part of REST's beauty stems from its use of standard HTTP methods. On an error, I emit a header with a 3xx,4xx or 5xx error status code. For a detailed error description, I can use the body (right?). So far so good. But what would be the way to transmit a proprietary error code that is more detailed in describing what went wrong (e.g. "failed to connect to database", or "database login wrong")? If I put it into the body along with the message, I have to parse it out afterwards. Is there a standard header for this kind of thing?

4. How to do authentication

• What would a API key based authentication following REST principles look like?
• Are there strong points against using sessions when authenticating a REST client, other than that it's a blatant violation of the REST principle? :) (only half kidding here, session based authentication would play well with my existing infrastructure.)

EDIT: Thanks for all your great responses guys. I will be going through them tomorrow or the day after.

REST API error return good practices

I'm looking for guidance on good practices when it comes to return errors from a REST API. I'm working on a new API so I can take it any direction right now. My content type is XML at the moment, but I plan to support JSON in future.

I am now adding some error cases, like for instance a client attempts to add a new resource but has exceeded his storage quota. I am already handling certain error cases with HTTP status codes (401 for authentication, 403 for authorization and 404 for plain bad request URIs). I looked over the blessed HTTP error codes but none of the 400-417 range seems right to report application specific errors. So at first I was tempted to return my application error with 200 OK and a specific XML payload (ie. Pay us more and you'll get the storage you need!) but I stopped to think about it and it seems to soapy (/shrug in horror). Besides it feels like I'm splitting the error responses into distinct cases, as some are http status code driven and other are content driven.

So what is the SO crowd recommendation? Good practices (please explain why!) and also, from a client pov, what kind of error handling in the REST API makes life easier for the client code?

Best Practices for securing a REST API / web service

When designing a REST API or service are there any established best practices for dealing with security (Authentication, Authorization, Identity Management) ?

When building a SOAP API you have WS-Security as a guide and much literature exists on the topic. I have found less information about securing REST endpoints.

While I understand REST intentionally does not have specifications analogous to WS-* I am hoping best practices or recommended patterns have emerged.

Any discussion or links to relevant documents would be very much appreciated. If it matters, we would be using WCF with POX/JSON serialized messages for our REST API's/Services built using v3.5 of the .NET Framework.