Using LINQ and Reactive Extensions to overcome limitations in OData query operator

I was pleased to know that Netflix had OData API to query. The practical reason is obviously was to use the API to query for the movies I want to watch. Like I mentioned in my previous post, I will be using LINQPad 4 for querying purposes, because of its built-in capabilities for OData as well as for Rx.

One thing I discovered after playing around with OData is that not every query operator in LINQ is available in OData. For example the Netflix API has only for 4 operators which are

  1. Filter
  2. Skip
  3. Take
  4. Orderby

And also the query returns only 20 rows as the result for each request. So for example if I have to get 40 rows, on my first request  the server would return 20 rows and in my next request I would have to skip first 20 and take next 20 to get 40 rows. These are some of the limitations.

Here is what I wanted from Netflix, I wanted to movie listings that has an average rating greater than 3.5 ,ordered by release year descending and grouped by listings that are available for instant watch.  So that I can have one queue for movies that I want to watch online and another one that I can request via mail (the ones that is not available in instant watch).  And here is the query to do that


 var movies = from counter in (from e in Enumerable.Range(0,400) where e%20  == 0 select e).ToObservable()
 from movieTitle in Titles.Where (t => t.AverageRating > 3.5).OrderByDescending (t => t.ReleaseYear).Skip(counter).Take(20).ToObservable()
 select movieTitle;

var moviesILikeToWatch = from counter in movies
 group counter by counter.Instant.Available into g
 select g;
moviesILikeToWatch.Dump();

The first “from counter” query is to build the skip part, like I mentioned by default the  result returns only 20 rows I wanted 400 rows to achieve that I used the enumerable range to generate sequence that I can use for skipping in my next query. I could have very well used for loop to build this, but that is not what I want. I want to try and write terse code. These are actual calls to Netflix OData  API

http://odata.netflix.com/Catalog/Titles()?$filter=AverageRating gt 3.5&$orderby=ReleaseYear desc&$skip=0&$top=20
http://odata.netflix.com/Catalog/Titles()?$filter=AverageRating gt 3.5&$orderby=ReleaseYear desc&$skip=20&$top=20

In the below picture linqpad makes 20 calls to Netflix for getting 400 movie listings

The next line in the first  query “from movieTitle” is simple Linq query to get movies based on filter criteria along with skip and take. The reason for the second query is because the OData  API doesn’t provide a groupby operator and if I include it in my first query , Linqpad would try and convert it to OData specific  request which would fail. So essentially I am getting all the data from the server and then grouping it locally.

This wouldn’t have been possible without OData.

About Naveen
Write code.

3 Responses to Using LINQ and Reactive Extensions to overcome limitations in OData query operator

  1. Regarding operators I would agree that there are limitations. But for constraining return results count per request, I would consider this as a constraint. You now that you have limited amount of results to 20 rows, you can fit yourself with this constraint.

    You mentioned that NetFlix APIs only support filter, skip, take and orderby! does that mean some other OData Services might support more?! As far as I know no! limitations in operators like GroupBy not supported as well as aggregations are cascaded on all OData Services. And I think this is where OData Operation might fit to perform complex stuff and expose them as operations.

    Nice post and thank you for point me to it.

  2. I don’t understand why ToObservable is used here. I ran the query without the two calls to ToObservable, and everything worked exactly the same.

    • Alok Arora says:

      the ToObservable call is used to force evaluation then and there.

      by default the LINQ calls are lazily evaluated by the time the corresponding data is actually being rendered or used somewhere. so likewise it’s required in some of the cases where you are looping through the resultset to perform some operations having some complex operations such as UI elements are being created or anything else which needs to block some memory to perform required operation.

      it helps to avoid that memory load for longer while the loop source is being calculated.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: