Customizing Witty Twitter Client Part 1- using C# as Compiler Service


This is going to be a multipart blog post where I am going to be demonstrating how I have customized Witty twitter client for my need. I chose Witty because it is the only OSS .NET twitter client I know of.

One of the reasons for customizing is primarily using C# as compiler service to extend it for my needs dynamically. For example I follow this twitter list http://twitter.com/shanselman/programmers , it’s a cool list that Scott maintains, thanks to him. But there is one person in this list who keeps tweeting about weight loss, which I am least interested and I didn’t have control over it but to ignore, until now. And it is always fun to write  software for your daily needs, which for me saves time.  Thanks to Mono for C# as compiler service. FYI I have shown a simple usage of Mono Csharp compiler service in this post.

Here are things that I could get done by just spending few hours of my weekend time

Here is my the default witty

Filter the list by user name  which I am not interested in


new Func<Tweet, bool>(t => !t.User.Name.Contains("CNN"));

Here is the same without CNN

FYI the filter C# code is in the Filter Text box

The next filter criteria is to look for tweets that have hashtag as fsharp and also at least have one link


new Func<Tweet, bool>(t => t.HashTags.DefaultIfEmpty().Contains("#fsharp") && t.Urls.DefaultIfEmpty().Count() > 0);

And here is the filtered list

Be even crazier look for tweets that have hashtag as fsharp and at least one link and then open the link automatically

new Func<Tweet, bool>(t => {
 if (t.HashTags.DefaultIfEmpty().Contains("#fsharp") && t.Urls.DefaultIfEmpty().Count() > 0)
 System.Diagnostics.Process.Start(t.Urls.First().ToString());
 return t.HashTags.DefaultIfEmpty().Contains("#fsharp") && t.Urls.DefaultIfEmpty().Count() > 0; });

The browser with the link opened automatically.

It’s a hack and I shouldn’t be doing the above, but it does the job.

There is so much more possibilities. I could easily provide a save feature for these search scripts and reuse the same.  At the end of the series I will post the entire code in GitHub. But if you want to try it before that here are the simple changes I started doing to the code.

Extension Method for Compilation


static class Extensions
 {
 public static object Compile(this string code)
 {
 return Mono.CSharp.Evaluator.Evaluate(code);
 }
 public static void Run(this string code)
 {
 Mono.CSharp.Evaluator.Run(code);
 }
 }

The filter code


public bool TweetFilter(object item)
 {
 Tweet tweet = item as Tweet;

 // this will prevent the fade animation from starting when the tweet is filtered
 tweet.IsNew = false;
 try
 {
 Func<Tweet, bool> compare = (Func<Tweet, bool>)FilterTextBox.Text.Compile();
 return compare.Invoke(tweet);
 }
 catch(Exception ex)
 {
 Console.WriteLine(ex);

 }
 return true;
 }

The hashtags and Links weren’t part of the tweet class , but they were part of the UI class. So I had to bring them to the tweet class and populate them.

public IEnumerable<Uri> Urls
 {
 get
 {
 return links;
 }
 set
 {
 links.Clear();
 links.AddRange(value);
 }
 }
 public IEnumerable<string> HashTags
 {
 get
 {
 return hashTags;
 }
 set
 {
 hashTags.Clear();
 hashTags.AddRange(value);
 }
 }

And here is the code to populate the above properties

private IEnumerable<Uri> GetLinks(string text)
 {
 string[] words = Regex.Split(text, @"([ \(\)\{\}\[\]])");
 return (from word in words
 let isUrl = new Func<string, bool>(s => UrlShorteningService.IsUrl(s))
 where isUrl(word)
 select new Uri(word) ).ToList();
 }
 private IEnumerable<string> GetHashTags(string text)
 {
 string[] words = Regex.Split(text, @"([ \(\)\{\}\[\]])");
 return (from word in words
 let ishash = new Func<string, bool>(s => s.StartsWith("#"))
 where ishash(word)
 select word).ToList();
 }

I know for the above 2 functions I could have written a High-Order Function

C# – Practical usage of High-Order Functions


I happened to write this code sometime back that took Excel data and converted it to specific xml format. It was a simple Linq statement and here is the code

from row in table.Rows.Cast<DataRow>()
     where row[0] != DBNull.Value &&
       row["length"] != DBNull.Value
     select new XElement("",new [] {
          new XElement("Number",row.<Double>("schema ")),
          new XElement("Name",row.<string>("name")),
          new XElement("Type",row.<string>("type")),
          new XElement("StartPosition",row["new loc"]),
          new XElement("Length",row["length"]),
          new XElement("PathQuery",row["Path"]),
          new XElement("Attribute"),
          new XElement("format")};

A new change request came in for this. The existing code was applicable for most of the worksheets ,except if the worksheet had column “FOO” then the data should also be filtered by column FOO . And not every worksheet would have column FOO.

At work ,I am the guy who always advocates Functional Programming. Especially when this was already in FP style, I was supposed to make less change to fulfill the requirement. The key is also DRY (Don’t Repeat Yourself). And here is my change to the function

Func<Func<DataRow,bool>,IEnumerable<XElement>> func = f =>
from row in table.Rows.Cast<DataRow>().Where(f)
     where row[0] != DBNull.Value &&
       row[" length"] != DBNull.Value
     select new XElement("",new [] {
          new XElement("Number",row.<Double>("schema ")),
          new XElement("Name",row.<string>("name")),
          new XElement("Type",row.<string>("type")),
          new XElement("StartPosition",row["new loc"]),
          new XElement("Length",row[" length"]),
          new XElement("PathQuery",row["Path"]),
          new XElement("Attribute"),
          new XElement("format")});

With the above code , I have parametrized the additional where clause which is  a high-order function. Remember ,I said this should continue working for excel sheets that does not have a column FOO and here is the code to satisfy both of them

Func<bool,string,Func<DataRow,bool>> filterData = (b,s)
=> dr => !b ? 1==1 : dr["FOO"].ToString().Trim() == s;

The filterdata would be the parameter for “func”. The filterData is again another high-order function that takes bool,string which returns Func<DataRow,bool>.

var isFOOPresent = table.Columns.Contains("FOO");
func(filterData(isFOOPresent,"bar"));

So now for the excel sheet that does not have column foo , the filter condition is 1==1 ,which works for the rest of the sheets. And for the sheet that has column FOO it is filtered by value “bar”. I was able to do this by changing just 4 lines of code. This is the reason I could convince rest of my team to think more functional.

Flattening List in F# and C#


I was teaching someone F# and comparing it with C#.  I had to bring in similarities between F# and C# and one of the key things List module. I was showing List Concatenation using “@” and also showing it C#. Here is an example of using it solving with selectmany


var x = new [] {new [] {10,20,30},new [] {40,50,60}};

x.SelectMany (y => y).ToList().ForEach(Console.WriteLine);

F# Solution


[10;20;30] @ [40;50;60]

Someone threw in a challenge by asking what if the list contains values and as well as other lists. Like example


var list = new List<object>();

list.AddRange(new object [] {14,new List<int>(),new List<int>() {1,10,23}});

And here was the code someone came up with in C#

list.Flatten<int>().ToList().ForEach(Console.WriteLine);

static class Extensions
{
 public static IEnumerable<T> Flatten<T>(this IEnumerable<Object> List)
 {
 var l = new List<T>();
 foreach (var element in List)
 {
 if (element is T)
 l.Add((T) element );
 else
 l.AddRange(((IEnumerable)element).Cast<T>());
 }
 return l;
 }
}

First of all the code isn’t type safe. I know there is a better way to solve in C#, but I wanted to show how this can be solved in a terse manner with F#

And here is my solution to above problem using in F#

type  T<'t> =
 |L of 't
 |N of T<'t> list

let rec flatten l =
 match l with
 |L x -> [x]
 |N x -> List.concat(List.map flatten x)

 let tree = N[L 14;N[];N[L 1;L 10;L 23]]
 let result = flatten tree

Using C# Compiler as a Service in F#, PoshConsole (Powershell)


I saw this amazing video from Anders Hejlsberg on C# as Compiler service and after which, Miguel de lcaza from the Mono posted about their implementation of C# Compiler service. Thanks Miguel .The best part is ,it works out of the box in Windows on the Microsoft .NET code base. So here I am using Mono’s C# compiler code  (Mono.CSharp.dll) within Microsoft .NET Code

This is cool because we could use C# more like python. And what if VS.NET can be customized using C# based scripting ,like Lisp for Emacs.

I figured out from Miguel’s post the only class we should be consuming is Evaluator. And here is a simple extension method for string class to compile and run.

static class Extensions
{
 public static object Compile(this string code)
 {
 return Evaluator.Evaluate(code);
 }
 public static void Run(this string code)
 {
 Evaluator.Run(code);
 }
}

And here is a code that uses the above extension, which creates a Func dynamically and passes it back to static code.

static void Main()
{

 var people = new List<Person>() { { new Person() { Name = "Bush", Age = 63, Sex = 'M' } },
 { new Person() { Name = "Obama", Age = 53, Sex = 'M' } }, { new Person() { Name = "Gordon", Age = 57, Sex = 'F' } } };
 Evaluator.Init(new string[0]);
 Evaluator.ReferenceAssembly(typeof(Person).Assembly);
 "using System;".Run();
 "using System.Linq;".Run();
 "using TestPerson;".Run();
 var whereexpression = (Func<Person,bool>)"new Func<Person,bool>( (p) => p.Age < 60);".Compile();
 people.Where(whereexpression).Dump();
}
class Person
{
 public string Name { get; set; }
 public int Age { get; set; }
 public char Sex { get; set; }
}

Here is a code that uses C# within F#

open System
open System.Collections.Generic
open Microsoft.FSharp.Core
open Mono.CSharp

Evaluator.Run("using System;") |> ignore
Evaluator.Run("using System.Linq;") |> ignore

let eval code =  Evaluator.Evaluate code

let list = ["FSharp";"CSharp"]
let code =  eval "new Func<string,bool>( (s) => s == \"CSharp\") ;" : ?> Func<string,bool>
printfn "Does list contain Foo? %b" (code.Invoke("Foo"))
printfn "Does list contain CSharp? %b" (List.exists (FuncConvertExtensions.ToFSharpFunc (code)) list)

let seqCast : seq<char> = Seq.cast (eval("from x in \"Bar\" select x;") : ?> IEnumerable<char>)
printfn "Sequence result:  %b" (Seq.exists(fun x -> x = 'c') seqCast)

I know there is F# powerpack for Linq. One of the reasons we would probably extend this is, if an application is developed in F# (targeted at Devs) and it could possibly allow extensions in C#.  Here is an example ,Seesmic has platform for building extension and how would it be If I could write a “ Func<Tweet,bool>( tweet => tweet.Language == “English” && tweet.HasLink == true)”  in the search box and if the platform happened to developed in F# (which is not) it would allow the C# devs to extend it .

Another interesting use is using C# compiler service within PoshConsole . In Powershell we could use Add-Type for creating types on the fly but with is I wouldn’t have to create a class and method we could get away with just creating lambdas. Here is an example of the usage

[Reflection.Assembly]::LoadFile("D:\tools\Mono.CSharp.dll")
[Mono.CSharp.Evaluator]::Run("using System;")
[Mono.CSharp.Evaluator]::Run("using System.Linq;")
$func = [Mono.CSharp.Evaluator]::Evaluate("new Func<int,bool>( (s) => s == 10);")
$func.Invoke(100)
$func = [Mono.CSharp.Evaluator]::Evaluate("from x in new [] {100,200} select x;")
$func

Imperative to Declarative in C#


One of things that I am trying to do is use try and create immutable objects primarily for concurrency and also helps in TDD.  So here is a sample code for the Immutable object


public class Foo
 {
 private List<Tuple<bool, string>> list;
 private string _name;
 private Foo()
 {
 list =  new List<Tuple<bool,string>>();
 }
 public static Foo Copy(string Name, IEnumerable<Tuple<bool, string>> list)
 {
 var f = new Foo();
 f.list.AddRange(list);
 f._name = Name;
 return f;
 }
 public string Name
 {
 get
 {
 return _name;
 }
 }
 public IEnumerable<Tuple<bool, string>> List
 {
 get { return list; }

 }
 public override bool Equals(object compare)
 {
 var compared = (Foo)compare;
 return Name == compared.Name;
 }
 }

There could be multiple of these objects in a list. Here is code for creating these items.


var first = Foo.Copy("first", new List<Tuple<bool, string>>() { Tuple.Create<bool, string>(false, "first") });

var second = Foo.Copy("second", new List<Tuple<bool, string>>() {Tuple.Create<bool,string>(false,"second")});

var firsttrue = Foo.Copy("first",new List<Tuple<bool, string>>() { Tuple.Create<bool, string>(false, "firsttrue") });

var secondtrue = Foo.Copy("secondtrue",new List<Tuple<bool, string>>() { Tuple.Create<bool, string>(true, "secondTrue") });

var firstList = new List<Foo>() { first, second };
var secondList = new List<Foo>() { firsttrue, secondtrue };

And here is the representation of the multiple lists

List<Foo> (2 items)
Name List
first
List<Tuple<Boolean,String>> (1 item)
Item1 Item2
False first
second
List<Tuple<Boolean,String>> (1 item)
Item1 Item2
False second
List<Foo> (2 items)
Name List
first
List<Tuple<Boolean,String>> (1 item)
Item1 Item2
False Firsttrue
secondtrue
List<Tuple<Boolean,String>> (1 item)
Item1 Item2
True secondTrue

The requirement is to join these lists into a single list based on Name. But when merging the list the tuple values should also be merged. The result after merging should be like this.

IEnumerable<Foo> (3 items)
Name List
second
List<Tuple<Boolean,String>> (1 item)
Item1 Item2
False Second
secondtrue
List<Tuple<Boolean,String>> (1 item)
Item1 Item2
True secondTrue
first
List<Tuple<Boolean,String>> (2 items)
Item1 Item2
False First
False Firsttrue

The imperative code used to achieve this was


var existslist = new List<Foo>();
 foreach (var element in firstList)
 {
 foreach (var e in secondList)
 {
 if (element.Name == e.Name)
 {
 existslist.Add(Foo.Copy(element.Name,element.List.Concat(e.List)));
 }
 }
 }
 for (int i = firstList.Count - 1; i >= 0; i--)
 {
 if (existslist.Contains(firstList[i]))
 firstList.RemoveAt(i);
 }
 for (int i = secondList.Count - 1; i >= 0; i--)
 {
 if (existslist.Contains(secondList[i]))
 secondList.RemoveAt(i);
 }
 firstList.Concat(secondList).Concat(existslist);

I certainly wasn’t happy after looking at the code. The idea behind creating immutable objects was going into more functional way of writing succinct code.

Here is the same thing in a much more functional approach.

var filtered = (from fst in firstList
                join snd in secondList on fst.Name equals snd.Name
                select Foo.Copy(fst.Name, fst.List.Concat(snd.List))).ToList();

firstList.Union(secondList).Where(l => !filtered.Contains(l)).Concat(filtered).Dump();
Follow

Get every new post delivered to your Inbox.

%d bloggers like this: