valerio.net

It’s standard practice for C# applications and assemblies to have metadata attached to the assembly specifying the version number, name, company, author, etc. Usually this information is specified in the Properties\AssemblyInfo.cs file.

However, F# libraries (by default) don’t have this assembly metadata, so you don’t know what version of a DLL you are looking at.  For example, without assembly information, MyLibrary looks like this in the Explorer window:

That’s not very helpful.

Thankfully, it’s very easy to add an AssemblyInfo.fs file to our F# library and achieve the same thing as a C# library.

Here’s the AssemblyInfo.fs file that you can copy/paste into your project.

To use it, make AssemblyInfo.fs the last file in your F# Library (DLL) project (remember – file order is important for F# projects).  If necessary, right click on the file and use the “Move Up” and “Move Down” commands.

After compiling the F# Library, it will then have all of the usual CLR metadata compiled into the assembly and show up in the Explorer window correctly:

It is very important to note that the last line of AssemblyInfo.fs is “()”.  The file won’t compile without this, since attributes (even though they are assembly-level attributes) must be applied to something.  In this case, we’re applying all of the attributes to the “nothing” placeholder in F#, called “unit”, or “()”.

It would be nice if the F# templates that ship with the F# CTP included this file by default. I guess I could always make my own template

Hope that helps!

This week at the F# for Scientists Book Club we got onto the topic of Project Euler, problem 11. 

To restate the problem:

In the 20×20 grid below, four numbers along a diagonal line have been bolded.

08 02 22 97 38 15 00 40 00 75 04 05 07 78 52 12 50 77 91 08
49 49 99 40 17 81 18 57 60 87 17 40 98 43 69 48 04 56 62 00
81 49 31 73 55 79 14 29 93 71 40 67 53 88 30 03 49 13 36 65
52 70 95 23 04 60 11 42 69 24 68 56 01 32 56 71 37 02 36 91
22 31 16 71 51 67 63 89 41 92 36 54 22 40 40 28 66 33 13 80
24 47 32 60 99 03 45 02 44 75 33 53 78 36 84 20 35 17 12 50
32 98 81 28 64 23 67 10 26 38 40 67 59 54 70 66 18 38 64 70
67 26 20 68 02 62 12 20 95 63 94 39 63 08 40 91 66 49 94 21
24 55 58 05 66 73 99 26 97 17 78 78 96 83 14 88 34 89 63 72
21 36 23 09 75 00 76 44 20 45 35 14 00 61 33 97 34 31 33 95
78 17 53 28 22 75 31 67 15 94 03 80 04 62 16 14 09 53 56 92
16 39 05 42 96 35 31 47 55 58 88 24 00 17 54 24 36 29 85 57
86 56 00 48 35 71 89 07 05 44 44 37 44 60 21 58 51 54 17 58
19 80 81 68 05 94 47 69 28 73 92 13 86 52 17 77 04 89 55 40
04 52 08 83 97 35 99 16 07 97 57 32 16 26 26 79 33 27 98 66
88 36 68 87 57 62 20 72 03 46 33 67 46 55 12 32 63 93 53 69
04 42 16 73 38 25 39 11 24 94 72 18 08 46 29 32 40 62 76 36
20 69 36 41 72 30 23 88 34 62 99 69 82 67 59 85 74 04 36 16
20 73 35 29 78 31 90 01 74 31 49 71 48 86 81 16 23 57 05 54
01 70 54 71 83 51 54 69 16 92 33 48 61 43 52 01 89 19 67 48

The product of these numbers is 26x63x78x14 = 1788696.

What is the greatest product of four adjacent numbers in any direction (up, down, left, right, or diagonally) in the 20×20 grid?

We started down the path of how to extract all of the possible sequences of 4 numbers from the 20×20 grid using sequence expressions.  However, it occurred to me that pattern matching might be a good fit for this, so I decided to try out an alternative approach using pattern matching.

The first step is to create a pattern-matchable structure containing our 20×20 grid of numbers. I decided to use an “int list list” — a list of a list of integers.  This is easy enough to do if we split the text into lines, then split each line into number strings, and then convert each number string into an integer:

In F#-ese, we take the text, and “pipe” (using the |> operator) it into the String.split operation that splits the string on newlines.  This gives us a list of strings (string list) that we can then “pipe” into a List.map operation that applies the given function to each string in the list (each row).  This mapping function splits each row string on spaces and then converts each number string to an integer.  I’m using function composition here (the >> operator), since the string split happens, and then the integer conversion happens.

As we’re hunting through all of the possible 4-number combinations (left-right, up-down, diagonal-left, and diagonal-right), we need to store some information.  We need 1) the product of the 4 numbers and 2) the 4 numbers themselves.  This seems like a good place to use a tuple, where we can store two things — the product and the 4-number list.  We can write a function that takes 4 numbers and creates this tuple for us:

The signature of this method is “val makeTuple : int -> int -> int -> int -> int * int list”, which says that we take 4 integers and create an (int * int list), i.e. a 2-element tuple with an integer and an integer list.

Now we come to the pattern matching aspect of the problem.  We need to apply a 4×4 filter to the 20×20 cell array, look for 4 patterns, and then repeat the process for the remaining cells.

Pattern matching in F# can easily dissect a list into its head (first element) and its tail (whatever’s left). This is denoted as “head::tail”.  You can also match with more than one element out of the front of the list, using “h1::h2::h3::tail”. In addition, you can ignore things if you don’t want to explicitly give them a name by using the “I don’t care” symbol, _, e.g. “h1::h2::_”.  And, you can group elements together and give them a new name, for example “h1::(h2::h3::_ as tail)”.

So, here’s the code for our pattern finding algorithm:

The findPatterns function takes our cells (a list of list of integers) and performs a pattern match over them.  The syntax is a little odd at first, so I made a picture to show what is matched with what:

If the cells match the pattern (that is, a 4×4 square has been found), then the 4-length sequences are stored as a tuple, and the method is recursively called again, twice. 

The first recursive invocation is called with a new 2-d list containing the 4 tails from the previous match. In this loop, [t1; t2; t3; t4] keep getting their leftmost elements stripped out until there isn’t a 4×4 square left, in which case the pattern match fails, _ is matched, an empty “list list” ([[]]) is returned, and the recursion stops.  So basically we are walking left-to-right matching patterns and storing the results.

The second recursive invocation is called with the tail t5.  This is the input cells without the first row.  With this invocation we’re recursively walking the cells from top to bottom.

So, once we are able to find all of the patterns, then to solve the problem we just need to find the maximum value of the tuples returned:

Here we invoke findPatterns with the cells (returning a list list), concatenate that down to a list (of int * int list), and then find the maximum using the first value of the tuple (the product of the 4 numbers).

The result is:

> result;;
val it : int * int list = (70600674, [87; 97; 94; 89])

Cool!  It doesn’t take any time at all to execute on my machine — well below the “1 minute rule” for Project Euler.

I’m sure there are ways to improve this algorithm.  It uses the stack quite heavily, and the list concatenation (@ operator) is expensive. Also, storing all of the results for all combinations wastes memory — a fold would be more efficient, allowing you to compare as you move along.

Despite these potential problems, I really like this solution because it is so concise.

After the awesome F# for Scientists Book Club meeting the other night, I wanted to set up a code repository for everyone to share their code.

After some discussion on the Google Groups list, we decided to use Git instead of Subversion for the source control.  Sure, why not? Might as well try out something new.  We found GitHub.com which offers free hosting for public projects.

Installing the Git Client

I hunted around for a TortoiseSVN-esque Windows client for git, but there isn’t one (yet?).  Ah well, back to the command line with Cygwin.  I usually have Cygwin installed, but my installation on this machine is broken for whatever reason.  Thankfully I stumbled onto msysgit, an all-in-one installer.

I ran the installer, using all of the default options:

Git uses SSH under the covers to connect to the remote git repository, so we need to spend a second setting up our public/private key pairs.

Double-clicking on the “Git Bash” shortcut on my desktop brings me up to a MINGW32 command prompt.  I need to generate new SSH keys, so I did:

$ ssh-keygen.exe

This generated a few files:
~/.ssh/id_rsa = Private key
~/.ssh/id_rsa.pub = Public key

You’ll need to cat the id_rsa.pub file and copy/paste that public key (starting with the “ssh-rsa” line) into your GitHub account next.

Creating a GitHub account

Just sign up for a free Github account (pretty self-explanatory).

You’ll need to provide the SSH public key (not the private key!) to github so that you’ll have permission to commit. You can enter the key on the first screen (it’s optional), or add one once you’ve created the account.

Using Git

I just followed the directions on the fsharpbooksamples project page.

$ cd /d/OpenSource/
$ mkdir fsharpbooksamples
$ cd fsharpbooksamples/
$ git init

Here I created and edited a README.txt file

$ git add README.txt
$ git config –global user.email matt.valerio@gmail.com
$ git commit -m ‘Initial commit’
$ git remote add origin git@github.com:mattvalerio/fsharpbooksamples.git
$ git push origin master

And, that’s about it.  Note that on the last push command, it will fail if you don’t have your SSH keys set up correctly. It will complain with an error message that says “Permission denied (publickey)”.  If you set up a passphrase while generating your SSH keys, you’ll need to enter that here.

Hope that helps!

This weekend I wanted to throw together a nice sample illustrating some ideas that I was playing around with this summer. While I was out at Microsoft Research, I got turned onto Microsoft Robotics Developer Studio and the incredibly cool Concurrency and Coordination Runtime (CCR) that gives it a solid foundation.

Interacting with remote services is difficult.  WCF has made the interaction easier using the “address-binding-contact” abstractions, but this doesn’t address things like concurrency , multithreading, asynchronicity, failures, and timeouts. Writing code that handles all of the possible scenarios can be messy very quickly. ThreadPools, BeginInvoke, EndInvoke, IAsyncResult, ManualResetEvent, and friends aren’t exactly the easiest thing to get right, but if you want a responsive UI and a robust application, you must worry about all of these things.

George Chrysanthakopoulos and the creators of the CCR recognized this and have provided an amazing abstraction on top of the current threading implementations on the .NET platform. I’ve blogged about the CCR before, but haven’t really posted any useful examples that truly illustrate its power. Get ready to be blown away — I’m going to show how easy it is to write a WinForms client that accesses a remote WCF service that:

• Correctly interacts with the UI on the correct thread
• Has full responsiveness of the UI
• Caches requests to the service if necessary
• Never blocks a thread while waiting for something, performing all operations asynchronously
• Never uses the “lock” keyword

I couldn’t believe how easy it was to get everything working.

The WCF Service

I’ve just made an extremely simple WCF Service for this demo.  It has a single operation, DoSomething that takes a request and generates a response. I like the way that the DoSomethingRequest and DoSomethingResponse allow for both multiple input and output properties on the respective types, making for a cleaner interface. Apparently I’m not the only one to think this, though the motivation is grounds for another complete post.

Our WCF service is contained within a WCF Service Library. This makes it easy to reference from other projects in the solution.

The request/response classes are very straightforward:

The implementation of the service is also very simple:

The service just takes in a request (an integer) and returns a response (a string) with more text.  However, if the input is 2, then an exception is thrown, so we’ll need to be ready on the client to handle this. (We’re ignoring fault contracts now for simplicity).

I should also point out that this service has InstanceContextMod=PerCall.  This means that WCF instantiates a new instance of the UsefulService class for every call that comes in. Also, we have ConcurrencyMode=Multiple, indicating that WCF also uses a new thread for every incoming call (with a preset maximum number that I can’t remember at the moment).  What this does is essentially blow the WCF service host wide open in terms of concurrency, allowing multiple threads to interact with their own instance of the class.  Since we’re not doing anything requiring synchronized access to shared state, these settings are largely unimportant.

Note that I’m not using the CCR on the service side — not yet.  I have some examples of how to do this, but that will unfortunately have to wait for future post Now onto the client…

The WCF Client

On the client side (a WinForms project), we add a reference to the service, making sure to go into the advanced option and turn on the generation of asynchronous operations.

This automatically generates a UsefulServiceClient class that not only has synchronous methods DoSomething, but also a method/event pair that can be used to invoke the operation asynchronously:

We’ll use the method/event pair for this example and won’t worry about using the Begin/End methods.

<Sidenote>

It’s also interesting to note that our generated IUsefulService interface on the client actually has 3 methods instead of 1 (with the attributes, cruft and unnecessary namespaces removed):

We could also use the {Begin/End}DoSomething pair here (also exposed on the UsefulServiceClient) since WCF can handle the APM (Asynchronous Programming Model pattern of the framework) invocation under the covers.  I’ll dig into this in a future post when we get to using the CCR on the server side. If you’re interested, Dan Rigsby has an excellent series of posts about asynchronous WCF service implementation. I’ll talk about this more in a future post.

</Sidenote>

CCR-Compatible WCF Client Wrapper

I’m not going to go into much detail here (check out this post to get started) about the architecture of the CCR. Suffice it to say that we have our very own thread pool, the DispatcherQueue, not related in any way to the .NET ThreadPool.  We create any number of “ports” which are just strongly-typed queues that can have messages “posted” to them.  Then, we can hook up handlers that get called whenever a message gets posted to the port.  The sender and receiver are completely decoupled, and the thread which executes the handler is managed by the dispatcher.

The key thing to take away from this is that ports (aptly named) are the “interfaces” between the producer and the consumer.  So, if we want to communicate with a service, we ask it for a port with which to post incoming requests for it to operate on.  On a similar token, we can also request a reference to a port with which we can hook up a handler to be called whenever the service obtains a response.

So, we can create a wrapper class around the auto-generated WCF client class that exposes two ports — an input (request) and an output (response) port.

Well, that’s the idea anyway.  Unfortunately we’ve subtly overlooked our failure scenarios.  Another possible outcome of a call to the service is that an exception is thrown.  The CCR allows us to create a conglomerate of ports called a PortSet which allows us to specify that either of two message types can be posted there.  We need this for the output port since the outcome of our operation could either be a response or an exception. This is a great approach since the failure information is explicit — we don’t have to wonder if an exception will just “bubble up” later.

Another point that needs to be made is that the CCR operates under a message-passing paradigm which emphasizes statelessness. Every message is self-contained.  So, we could post message requests into a port and wait for message responses on another port, but because of the asynchronous nature of things, we can’t assume that the response order will match the request order.  If we need to know which request generated which response (as is the case here), we need to be more explicit. The solution isn’t difficult — we just bundle a request with its response and call it a DoSomethingContext:

This brings us to the first glimpse of our UsefulServiceWrapper class:

We notice that our wrapper inherits from CcrServiceBase, a lightweight base class that just provides a TaskQueue property that exposes a DispatcherQueue which is set in the constructor, as well as a few other convenience methods.  We need to have access to the application’s DispatcherQueue to be able to send and receive messages through the ports.

There are three ports (or Ports/PortSets to be particular) here.  The first port is the input to the DoSomething operation which takes a DoSomethingContext (in this case the Request property will have a non-null DoSomethingRequest and the Response property will be null). The second port is a PortSet of a DoSomethingContext and an Exception, indicating that there are 2 possible outcomes of the operation.  The third port is a convenience port which allows the internal workings of the wrapper class to be observed (if desired) by allowing log messages to be posted to it.

Wrapper Logic

While implementing the wrapper logic, we have one rule — NEVER block a thread while we’re waiting for WCF to do something.  If we’re not doing something constructive with the thread (a scarce resource), then let the CCR execute other code with it.

Consequently, the standard WCF client convention is completely out:

This ties up a single thread during the entire execution of the operation.  If there’s a lot of network delay, I wouldn’t be surprised if 99% of the time the thread is sitting there idle, waiting for a response. This really limits our scalability in a big way. This isn’t going to work work — we need to use the asynchronous equivalent for everything — {Begin/End}Open, DoSomethingAsync/DoSomethingCompleted, and {Begin/End}Close.

Let’s take a look at just the constructor operation.  We’re going to use an awesome feature of the CCR – CCR Iterators — that uses C# 2.0 iterators to write synchronous-looking code that executes asynchronously.  The trick is to implement a method that returns IEnumerator<ITask>.  ITask is just a “unit of work” for the CCR, and we return an enumeration.  We get to use the yield keyword inside of iterator methods.

When you think about it, iterators are methods that execute different code each time that you call them.  That is exactly why they are well-suited for doing asynchronous programming — the individual sections of the method can be executed by different threads in an asynchronous manner as they are needed. The compiler works its magic behind the scenes, actually implementing iterators as a state machine. Pretty cool stuff.

Here’s the first part of the implementation showing how we can execute the constructor and catch any exceptions:

First we create a SuccessFailurePort, just a fancy name for PortSet<SuccessResult,Exception>. Then we try to call the constructor inside of a try/catch block. If the constructor completes successfully, then we post a success result to the success/failure port.  If an exception occurs, then we post the exception to the port.  The next line is where the magic of the CCR comes in — we “yield return Arbiter.Choice(successFailurePort…)”.  Translated into english, this means: “Return this thread back to the CCR and don’t come back until EITHER a success result OR an exception gets posted to the successFailurePort. Then, when we’ve ‘joined’ up with either result, execute either of these next two anonymous methods depending on which one was posted.”  Whoa.  That’s an amazing amount of power right there, after you get used to the unfamiliar syntax.

So, If there is a successful result, a message is posted to the log and the method continues down.  If there is an exception, we post a message to the log and also post the exception to the DoSomethingOutputPort, and set the failed flag to true.  If that flag is set, then we “yield break”, indicating to the CCR that we are completely finished with this method and never expect to re-enter the iterator. When we yield break, we’re completely done.

(Bear with me — even though this looks like a lot of code, it’s boilerplate. I’ve written a T4 template for generating it.  Also, there are less verbose ways of calling things (thanks to some extension methods in the latest version) but they can be difficult to grasp without first seeing things like this. We’ll see soon that interacting with this code via the ports is a breeze, so it’s worth it to write it once and forget about it.)

Now, assuming that the constructor didn’t throw an exception, we can now open the client (asynchronously of course):

This pattern should look familiar — it is the standard method of interacting with APM code.  The trick is that the EndXXX method can always throw an exception if the operation failed, so we need to handle that in the same way as before with the successFailurePort. Notice that if we have two possible paths (e.g. success or failure) then both paths must be capable of posting something to the port to be able to continue past the Choice.

Now we can interact with the DoSomethingAsync operation.  First we subscribe to the even handler, call the method, and then wait for either success or failure:

Once again, if there was a failure then the exception is posted to the output port and we “yield break” indicating that the iterator method has completed (completely).

As the last step, we then call {Begin,End}Close:

If we make it this far in the iterator method, then we are finished and a successful result has been stored in the DoSomethingContext.  We can then post the completed context (containing both the request and the response of the DoSomething method) to the output port:

And…we’re done. Almost.

The only thing missing is how we specify that DoSomethingIterator contains all of the code to run when something is posted to the DoSomethingInputPort.  To do this, we need to “activate” a “receiver” on the input port — that is, wire up the input port so that it runs the iterator. In this case, it’s as easy as putting this in the constructor:

The TaskQueue property refers to the DispatcherQueue given to the wrapper class’s constructor.

Now that we have the verbose part out of the way, let’s take a look at how we can use our CCR-compatible wrapper class by interfacing through its ports.

The WinForms Client

Ok, now we get to the fun part. So far we have implemented a service, auto-generated a WCF proxy to the service, and created a wrapper class around the WCF proxy.  Now all we need to do is create a form to illustrate how to use our CCR WCF wrapper class.

The form is simple: 1 input text box, 1 button, and a scrolling output text box.

When I press the Send button, I want to fire off N simultaneous requests to the WCF service with the request payload counting from 1 to N. When a request comes back, I want to log some information in the bottom text box.  I also want to handle all service exceptions, and I want to be able to move the window around while the service calls are being executed without having redraw problems.

Ok, so we’re faced with an interesting scenario here: to be CCR-compatible, we need to use the CCR  DispatcherQueue (essentially a thread pool), but how can we do that when WinForms has very specific threading requirements to handle the underlying Win32 API interop (message pump, etc)?

The trick is that the CCR comes with a number of WinForms adapters that make this impedance mismatch easier to handle.  Unfortunately I didn’t find everything that I needed in the Microsoft.Ccr.Adapters.WinForms namespace, so I needed to write a little more interface code.  The result is a programming API that directly mimics a typical WinForms program.  Normally the Program.cs file contains:

However, I’ve written a CcrApplication.Run static method so that we can do this instead:

That’s not so bad.  The only trick here is that instead of passing in a Form argument to Application.Run is that we’re passing in a Func<DispatcherQueue,CcrForm> delegate.  This is just a method that takes a DispatcherQueue and returns a CcrForm.  What is this CcrForm, you ask?

It’s just an abstract base class that accepts a DispatcherQueue in the constructor and uses that queue to set everything up. The key here is that the CcrForm has a property called FormServicePort of type WinFormsServicePort that lets us interact with the form (create a new one, invoke delegates on the UI thread, and close down the form).

I modified the code from this post so that it correctly handles form shutdown. The CcrApplication.Run method just creates a DispatcherQueue and then initializes a ManualResetEvent that we can wait on to keep the main thread from exiting (otherwise our window wouldn’t even show). We also hook up to the FormClosed event on the form so that we can cleanly shut down.

The CcrForm class is also fairly straightforward, just storing the DispatcherQueue so that we can use it from the form.

The Activate method is just for convenience so we don’t need to keep typing this.TaskQueue.  The FormInvoke method also makes calling it easier (as we’ll see in a sec).

Client Logic

Ok, we’re almost there! We need to “wire up” the logic in the form:

• Whenever a message is posted to the log port, display the information in the bottom text box
• Whenever a message is posted to the service output port (successfully), display the result in the bottom text box
• Whenever a message is posted to the service output port (exception), display the exception message in the bottom text box
• Whenever the Send button is pressed, post messages to the service input port.

The first 3 are receivers, and then last 1 is a post.  We can set up the first three persistent receivers in the constructor like this:

The Log method is just a helper method that appends a line of text to the lower textbox by wrapping the code up into a delegate and posted it to the WinFormsServicePort so that it may be executed on the UI thread.

The only thing we have left is the button handler which posts new messages to the service input port:

There’s an arbitrary limit of 50 messages in there, just so you can’t go completely nuts (100,000 requests might be a bit much).

Testing

After all of that work, we can finally try things out! I set my solution so that both my WinForms project and my WCF service get started when I hit F5.

The WCF test client starts up, indicating our service is live:

Now we dial in 10 as the number of threads to start and click the Send button.  Not just once.  20 times.  I just totally wail on the Send button and watch what happens:

After about 20 seconds, all of the queued requests finally finish and all log lines get printed to the text box.  Amazingly, I was able to constantly move the window around my screen without any hiccups as all of that was going on! And, to think that all of that happened without writing a single line of lock synchronization code…

Conclusion

I hope you enjoyed this in-depth exploration into the amazing world of the CCR.  I know I sure did. I’m quickly becoming a fan of the CCR’s model of event-driven asynchronous programming, where you can define your interfaces between components and then just “wire them up” with receiver code.  It certainly makes for a clean model.

To recap, we were able to write clean code that correctly interacted with the UI thread, giving us a fully-responsive UI, were able to cache the service requests by posting them into the wrapper’s input port, and we NEVER blocked a thread while waiting for asynchronous I/O. In addition, all failure conditions were handled cleanly. The code might seem pretty complicated at first (lots of anonymous methods everywhere) but after you get used to the way things work, it’s very powerful.  The ability to write synchronous-looking code that executes asynchronously is (to put it simply) pure genius on George’s part.

One thing that I didn’t cover is the CCR’s ability to coordinate lock-style code (e.g. to access shared state).  I’ll take an in-depth look into coordination/synchronization, as well as how to use the CCR to implement asynchronous WCF services in a future post.

I’ll put up the code soon for everyone to download.

Hope that helps!

I was going through some old code and ran across this little gem of a MATLAB script, so I thought I’d post it here for posterity. 

The script lets you specify an image file to view and lets you click on each data point until you’re finished.  Then it exports the (x,y) coordinates of all of the points that you clicked.  This is indispensable if you have a graph from a paper that you would like to analyze further but don’t have the original data file.

% digitize.m
function [x,y] = digitize(imgname)

[img,map] = imread(imgname);
image(img);
colormap(map);

Left = input('Enter the left X coordinate: ');
Right = input('Enter the right X coordinate: ');
Bottom = input('Enter the bottom Y coordinate: ');
Top = input('Enter the top Y coordinate: ');

disp('Click on the lower-left corner');
[localLeft,localBottom] = ginput(1);
disp('Click on the upper-right corner');
[localRight,localTop] = ginput(1);

disp('Begin digitizing points. Click right mouse button when finished.');
b = 0; bold = 0;
while(b~=3)
   [xp,yp,b] = ginput(1);
   if (b == 1)
      if (bold ~= 0)
         line([xold; xp], [yold; yp]);
         x = [x; xp];
         y = [y; yp];
      else
         x = xp;
         y = yp;
      end
      xold = xp; yold = yp; bold = b;
   end
end

x = (x - localLeft) / (localRight-localLeft) * (Right-Left) + Left;
y = (y - localBottom) / (localTop - localBottom) * (Top-Bottom) + Bottom;

Credit goes to Brian Usner, wherever you are

I just ran across this article by Jeff Gould explaining that Microsoft has joined the Advanced Message Queuing Protocol (AMQP) working group. This is really exciting because AMQP is an open standard with a sizable number of cross-platform implementations. Jeff makes the analogy that:

“…AMQP is to high-value, reliable business messaging what SMTP is to e-mail.”

This certainly has the potential to really change the landscape for the better (and away from the proprietary messaging systems like IBM’s MQ or Tibco’s Rendezvous). The current 0.9 specification makes mention of different scales of deployment:

• “Developer/casual use: 1 server, 1 user, 10 message queues, 1 message per second
• Production application: 2 servers, 10-100 users, 10-50 message queues, 10 messages per second (36K messages/hour)
• Departmental mission critical application: 4 servers, 100-500 users, 50-100 message queues, 100 messages per second (360K/hour)
• Regional mission critical application: 16 servers, 500-2,000 users, 100-500 message queues and topics, 1000 messages per second (3.6M/hour)
• Global mission critical application: 64 servers, 2K-10K users, 500-1000 message queues and topics, 10,000 messages per second (36M/hour)
• Market data (trading): 200 servers, 5K users, 10K topics, 100K messages per second (360M/hour)”

Wow. That’s a lot of messages, with a lot of critical applications riding on top. They are also targeting a number of different system architectures:

• “Store-and-forward with many writers and one reader
• Transaction distribution with many writers and many readers
• Publish/subscribe with many writers and many readers
• Content-based routing with many writers and many readers
• Queued file transfer with many writers and many readers
• Point-to-point connection between two peers
• Market data distribution with many sources and many readers.”

There is a C#/.NET implementation called RabbitMQ that looks extremely interesting. There is even a version compiled explicitly for .NET 3.0 with WCF bindings.  There is even a good bit of documentation. Sweet.  I really need to try this out and see how easy it is to set up a publish/subscribe system with reliable messaging.

Since Microsoft released the charting controls used in SQL Server 2008 reporting as a standalone product, I decided to try them out from F#.

This makes a lot of sense because F# allows you to write scripts and run them from within Visual Studio in F# Interactive mode.  I love it because it’s the flexibility of MATLAB scripting (just try new things out without compiling) with the power and speed of the .NET Framework (MATLAB is based on layers and layers of Java and is by no means fast…).

To use the charting library from F#, the first thing we need is a form with a single chart control docked in it.  This is relatively simple, though I’ve copy/pasted/modified a lot of the visual designer code from one of the demo projects, so it looks like a lot, but it’s not.

At the end, I also expose the internal Chart, Legend, Title, and ChartArea objects.

With that out of the way, we need a few helper methods:

The first method, linspace, just creates a uniformly-spaced list of floating point values from start to stop.  MATLAB users recognize linspace without a doubt.

The populateSeries method takes equal-length lists xList and yList and goes through each pair and adds those points to the specified series.

With those out of the way, we can go to town!

Wow, that was pretty painless. We start by creating a ChartForm and setting a few properties like the title and size, set up the series display options, and then set up our x- and y-values of numbers according to the function y=2sin(x)+4sin(2x).  Then we populate the series with the values and show the form. You can’t get much simpler than that. I’ve reused the showForm method from my previous post.

Here’s the result:

Slick, eh?

Lately I’ve FINALLY been getting back into some F# after a brief hiatus.  I love the way that it keeps stretching my brain.  Things have changed quite a bit with all of the slick new usability enhancements in the latest CTP. I need to dig out my data analysis scripts from the beginning of the year.

One of the first things I tried to make when I was learning F# was a simple “Hello World” Windows Forms application.  Quickly you run into a problem — if you want to take advantage of the awesome “F# Interactive” window at the bottom of the screen, then after you have a form, you call .Show().  However, if you want the form to come up when the compiled application is executed, then you need to use Application.Run().  If we try to use Application.Run from within F# Interactive, we get the following error:

Hmmm, we need a way to distinguish whether we are in compiled or interactive mode.

Thankfully the F# team has defined a number of compiler flags named COMPILED and INTERACTIVE that we can use.  So we can have conditional code depending on the execution mode:

This works out well, since we can now just call showForm and not worry about what mode we’re in.  For example, this uber-simple Hello World WinForms application can be written as:

Then, to run it in F# Interactive, just hit Ctrl-A (select all) and Alt-Enter (run selection in the F# Interactive Window).  Bam! Similarly, to run it in compiled mode we just punch F5.

Hope that helps! This won’t be the last F# post, trust me.

Doing any kind of graph charting or data visualization in managed code has always required either expensive 3rd-party software (e.g. Dundas, etc) or feature-incomplete open source projects (ZedGraph and NPlot come to mind). Furthermore, doing COM interop with Excel charts can be a nightmare.

Many times I just want to plot a bunch of data, probably an X-Y plot, to see a trend in some blob of numbers I have laying around.  Being an engineer, most of the time I’ll reach for MATLAB, but lately I’ve been doing quite a bit of XML processing that (to say it nicely) is not at all well-suited for MATLAB. I wanted to stay on the .NET Framework, take advantage of the .NET libraries I’ve written to parse the files (Linq to XML anyone?) and plot the data.  I’ve used ZedGraph in the past as a managed charting library, but it can be quirky.

I just discovered yesterday that Microsoft has released Microsoft Chart Controls for Microsoft .NET Framework 3.5.  Whoa, really? Could this be a better solution to ZedGraph? They even have samples available for download. I decided to try it out.

I ran the installer (MSChart.exe) and found that the assemblies get installed to “C:\Program Files\Microsoft Chart Controls\Assemblies”. There are 4 dlls, 2 for WinForms and 2 for ASP.NET:

• System.Web.DataVisualization.Design.dll
• System.Web.DataVisualization.dll
• System.Windows.Forms.DataVisualization.Design.dll
• System.Windows.Forms.DataVisualization.dll

After installing the controls, I unzipped the samples and opened up WinFormsChartSamples.exe:

Wow, that doesn’t look familiar at all.   I could swear I’ve seen controls like that before….oh yeah, Dundas!  After a bit of poking around, it appears that Microsoft (specifically the SQL Server 2008 team) has acquired the data visualization bits from Dundas.  Cool. I’m glad that Microsoft released the stand-alone assemblies for charting so that I don’t have to install SQL Server 2008 just to get 2MB of DLLs

The charting controls are extremely customizable. Take a look at some of the samples:

Very cool.  I’ll probably give them a test drive here soon.

Hope that helps!

Recently a couple friends have asked me how to get started using Subversion to manage their files, so here’s a step-by-step guide to get everything up and running.

Why use Subversion?

Subversion lets you:

• Keep track of every version of a file stored in multiple locations (e.g. desktop, laptop, work machine, etc)
• Synchronize these files so that no changes are lost, no matter which machine the file is last edited on
• Collaborate with other people using the same set of files without losing changes no matter who is editing the files

During undergrad, I always dreaded group projects because everything typically involved the “divide and conquer” methodology.  My group members and I would divide up the work and then get together later and combine all of our work.  In theory this works great, but in practice, it becomes a major hassle, especially if multiple iterations are involved, because inevitably someone’s work would accidentally get erased or lost.

It wasn’t until I reached grad school (and learned how to use Subversion effectively) that things changed.  I had the opportunity to work on a number of group projects (like group papers and presentations) where all members used Subversion to coordinate the files.  The final crunch time was much more sane since all changes to the files from all group members could be tracked and merged seamlessly. It really was a night and day difference.

Subversion Architecture

To use Subversion, each “set of files” is called a “repository”.  A centralized “Subversion server” must be used, and may contain any number of file repositories. To access these files, any number of “Subversion clients” may be used, typically from different machines.  Since Subversion is open-source, a considerable amount of effort has been dedicated to making the system cross-platform.  In general a Subversion server may be set up on Linux, Windows, or Mac OSX, and Subversion clients exist similarly for Linux, Windows, and Mac.

When files are retrieved from the server to the client, it is called an “update”, and when new versions of the files are sent to the server from the client, it is called a “commit”.

A typical repository will go through a continuous cycle of update-edit-commit.

In this guide, I’ll explain how to set up both a server and client on Windows.  In my experience, the most user-friendly choice is VisualSVN as the Subversion server and TortoiseSVN as the Subversion client.

Install VisualSVN Server

• Go to the VisualSVN Server homepage
• Click on “download now“
• Download the latest version (1.6.1 at the time of this writing).  You should have received VisualSVN-Server-1.6.1.msi
• Execute the installer after it has finished downloading
• Accept the license agreement
• In the next screen, leave the default values:
  
• This will set up a directory C:\Repositories\, where each subdirectory here will be the root directory for that particular file repository.   VisualSVN Server will let you set different permissions on each repository (and subdirectories in each repository, but that’s more advanced.) 
• You’ll also want to use Subversion authentication (password hashes stored by VisualSVN Server) instead of Windows authentication (VisualSVN integrates with the existing Active Directory domain, if present — this is quite advanced as well.)
• Then just click Install.
• This will bring you to the main VisualSVN Sever management console:
  
• Notice that there are no repositories, no users, and no groups (yet)

Create Users

• First we’ll need to create a user that has permission to access the repositories (which ones we’ll assign later)
• Right click on Users and click “Create User…”
• Enter a username (e.g. “matt”) and a password (twice) and click OK:
  
• Advanced Note: If you have a number of users that will all have the same permissions, it may make sense to create a group and add all of those users to that group.  Then when adding permissions to the repositories, you can just add the single group instead of adding each user. (Not shown.)

Create Repository

• Right click on Repositories and click “Create New Repository…”
• Enter a (case-sensitive) repository name and click OK:
  
• Advanced Note: If you know how to organize and use the typical Subversion directory structure (trunk, branches, tags), then check the box to automatically create those subdirectories in your repository.  If you don’t know why you’d need those, then don’t worry about it

Set Repository Permissions

• Now we need to set the permissions of our newly-created Dissertation repository, so right click on Dissertation and click “Properties…”
  
• We probably don’t want to give everyone read/write access to our Dissertation (:P), so the first thing we do is select Everyone and click “No Access”.
• Now we need to give permission to ourselves, so we click “Add…”, select “matt” and then OK:
  
• This leaves us with the following screen, showing that user matt has read/write access to the Dissertation repository:
  
• This is exactly what we’re after!  Feel free to tweak to your own needs.  VisualSVN makes things incredibly easy to configure (as opposed to the alternative of hand-editing text files.)

Now we have the Subversion server completely installed, configured, and running.  The next step is to install a Subversion client so we can access our file repository.

Install TortoiseSVN

TortoiseSVN is the most useful Subversion client out there.  It is an “explorer addin” which adds functionality to your Windows Explorer windows.  When you check out a repository (a local copy of the files that you can edit) from the remote Subversion server, it shows the status of each file.  Unedited (current) files show up with green checkmarks beside them, files with changes have red X’s, and files not in the repository have question marks. Here’s an example:

You can see that for the most part, no files have been modified.  Only LocalSandbox.cs and some files in the Utilities directory have changed since the last version was checked in. The “bin” and “obj” directories have been intentionally excluded from the repository.

• Go to the TortoiseSVN download site
• Download the installer (TortoiseSVN-1.5.5.14361-win32-svn-1.5.4.msi at the time of this writing)
• Accept the license, keep the default settings, and click Install.
• After finishing the installation, it may ask you to reboot:
  
• Go ahead and click Yes to reboot (since TortoiseSVN gets loaded into Windows Explorer, it actually is crucial to reboot.)

Try Everything Out!

• Create a directory in C:\ (or wherever) called (for example) “RepositoryCheckout”.
• It is crucial that you make the distinction between the repository directory (C:\Repositories\* where the Subversion server stores the internal file repository databases) and the repository checkout directory (C:\RepositoryCheckout\*) where we store the local copy of the files as we’re editing them). The repository directory exists only on the server, and the repository checkout directory exists only on the client, but we have them on the same machine now since we have both the server and the client running.
• Make a subdirectory here called Dissertation:
  
• Right click on Dissertation and notice the new TortoiseSVN menu items (SVN Checkout and the TortoiseSVN submenu).
• Click on “SVN Checkout…”:
  
• Now we need to fill in the URL of our repository. To get this, we can back to our VisualSVN Server window.  Since you probably rebooted, it’s in Start > All Programs > VisualSVN > VisualSVN Server Manager. Right-click on the Dissertation repository and then click “Copy URL to Clipboard”. Paste this URL into the “URL of repository” textbox in the TortoiseSVN Checkout window.
• In my case the URL was “https://ESL0095:8443/svn/Dissertation/”, though https://localhost:8443/svn/Dissertation/ would have worked just as well. Click OK.
• Because we’re using a secure connection (https instead of plain-old http), TortoiseSVN will ask us if we want to validate an unknown SSL certificate:
  
• We trust the certificate (VisualSVN Server just installed a self-signed cert by default), so click “Accept permanently”.
• Now we’re prompted for our credentials, so enter the username and password that you configured in the VisualSVN Server management utility before:
  
• Make sure that the “Save authentication” checkbox is ticked so that you don’t have to enter your username and password for each and every operation in the future.
• This brings up the “Checkout Finished!” dialog box, showing us that our (empty) repository is a Revision 0:
  
• For good measure, let’s add a file to the repository and commit it.
• Create a file called README.txt in “C:\RepositoryCheckout\Dissertation”:
  
• Right-click on README.txt > TortoiseSVN > Add… and click OK, then OK:
  
• This will show a blue plus sign on the file, indicating that it was added to the local repository:
  
• Now we just need to commit our changes to the local repository back to the Subversion server, so go back up one directory to C:\RepositoryCheckout
• Right-click on the Dissertation folder and select “SVN Commit…”
• Enter a log message describing what changed in this commit:
  
• After clicking OK, TortoiseSVN will commit our changes to the server and, if successful, indicate that we now have “Revision 1″ of the file repository!
  

Conclusion

Well, that wraps it up!  Hopefully that is enough information to get started using Subversion to keep track of your files.  There are a lot more advanced topics I haven’t covered here — if you have questions, feel free to leave a comment.

The best way to learn SVN is just to set up a “sandbox” repository and play around   Create files, add files, edit files, commit files, delete files, update files — go so far as to create two folders containing checked-out files (simulating two machines) and mess around with things until you are comfortable and confident enough to trust it with all of your files.  It took me a couple hours or so before I felt ready.

If you want to dig deeper, the Subversion web site has a lot of good information. Also, there is a great book called “Version Control with Subversion” available for free online.

Hope that helps!