cat head > /dev/www From my head to the web

1May/120

CtrlP

Recently stumbled across this plugin for vim. It’s replaced my use of command-t and fuzzyfinder. CtrlP is able to find files or buffers and let you switch to them in an intuitive manner. It also keeps a list of most recently used files or buffers. For example entering the command :CtrlPMRU brings up the list of MRU buffers.

Filed under: programming, vim No Comments
25Apr/120

Bochs and a Kernel

There’s a basic tutorial here here on how to get started with a kernel. It uses Bochs as 32 bit emulator. However recent versions of Bochs have been updated so that this line in the config file:

romimage: file=/usr/share/bochs/BIOS-bochs-latest, address=0xf0000

must change to this:

romimage: file=/usr/share/bochs/BIOS-bochs-latest
Filed under: Uncategorized No Comments
12Dec/110

old website

Finally uploaded content of my old website here

Filed under: Uncategorized No Comments
18Sep/110

Expert F# Errata

It’s good when you find errors in text books. It means you understand the subject matter. On page 231 of Expert FSharp 2.0

attempt { let! n1 = failIfBig inp1
let! n2 = failIfBig inp2
let sum = n1 + n2
return sum };;

desugars to this:

attempt.Bind( failIfBig inp1,(fun n1 ->
attempt.Bind(failIfBig inp2,(fun n2 ->
attempt.Return sum)))))

but it should be this:

attempt.Bind( failIfBig inp1,(fun n1 ->
attempt.Bind(failIfBig inp2,(fun n2 -> let sum = n1 +  n2 in
attempt.Return sum)))))

I confirmed it on stackoverflow (http://stackoverflow.com/questions/6182055/is-this-a-correct-desugaring-of-the-computation-workflow) and via email with one of the authors (Don Syme).

10Jul/114

Code completion in Vim for iOS

I recently started doing iOS developement. It’s great to be starting something new and learning new things. I’ve enjoyed learning Objective-C although I have a strong C++ background. But one thing that is annoying is having to learn the key bindings of yet another IDE. I’ve had to learn the bindings of Visual Studio, Eclipse and this time it’s XCode. Why can’t I just use a single editor that I know well enough that works on all platforms to do my coding? And given that my work patterns at the moment require me to chop and change between developing on Windows to Linux to Mac OS X daily, the most irritating thing is having to switch editing environments. It’s really frustrating when you press a key combination that’s been wired to your muscle memory to find out it does the wrong thing because you’re not editing in the editor you thought you were. I know Vim fairly well and am a relatively fluent user of it so naturally I wanted to use it for iOS devlepement. But XCode has such nice code completion? Enter clang_complete. It’s a plugin for Vim that uses clang to perform code completion. clang is also used by XCode for the same reason. Eventually it will replace the gcc-llvm compiler. I downloaded and installed the script but it didn’t work for my objective-c code for iOS. By running xcodebuild in verbose mode to see what commands were passed through to the compiler I was able to come up with the command line arguments to configure clang_complete with.

CompileC build/CrimeBook.build/Release-iphonesimulator/CrimeBook.build/Objects-normal/i386/main.o CrimeBook/main.m normal i386 objective-c com.apple.compilers.llvmgcc42
    cd /Users/sashan/code/CrimeBook
    setenv LANG en_US.US-ASCII
    setenv PATH "/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin:/Developer/usr/bin:/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin"
    /Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/llvm-gcc-4.2 -x objective-c -arch i386 -fmessage-length=0 -pipe -std=gnu99 -Wno-trigraphs -fpascal-strings -Os -mdynamic-no-pic -Wreturn-type -Wunused-variable -isysroot /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator4.3.sdk -fexceptions -fasm-blocks -mmacosx-version-min=10.6 -gdwarf-2 -fvisibility=hidden -fobjc-abi-version=2 -fobjc-legacy-dispatch -D__IPHONE_OS_VERSION_MIN_REQUIRED=40300 -iquote /Users/sashan/code/CrimeBook/build/CrimeBook.build/Release-iphonesimulator/CrimeBook.build/CrimeBook-generated-files.hmap -I/Users/sashan/code/CrimeBook/build/CrimeBook.build/Release-iphonesimulator/CrimeBook.build/CrimeBook-own-target-headers.hmap -I/Users/sashan/code/CrimeBook/build/CrimeBook.build/Release-iphonesimulator/CrimeBook.build/CrimeBook-all-target-headers.hmap -iquote /Users/sashan/code/CrimeBook/build/CrimeBook.build/Release-iphonesimulator/CrimeBook.build/CrimeBook-project-headers.hmap -I/Users/sashan/code/CrimeBook/build/Release-iphonesimulator/include -I/Users/sashan/code/CrimeBook/build/CrimeBook.build/Release-iphonesimulator/CrimeBook.build/DerivedSources/i386 -I/Users/sashan/code/CrimeBook/build/CrimeBook.build/Release-iphonesimulator/CrimeBook.build/DerivedSources -F/Users/sashan/code/CrimeBook/build/Release-iphonesimulator -DNS_BLOCK_ASSERTIONS=1 -include /var/folders/vx/vx-91r-0HPqbIKIRlmQhbk+++TI/-Caches-/com.apple.Xcode.501/SharedPrecompiledHeaders/CrimeBook-Prefix-bemjhtzaxsuzvgcgukqqlhmzivkm/CrimeBook-Prefix.pch -c /Users/sashan/code/CrimeBook/CrimeBook/main.m -o /Users/sashan/code/CrimeBook/build/CrimeBook.build/Release-iphonesimulator/CrimeBook.build/Objects-normal/i386/main.o

Stripping away much of the above by trial and error revealed the pertinent options to pass to clang that would allow a single file to compile were:

clang -fblocks -isysroot /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator4.3.sdk -D__IPHONE_OS_VERSION_MIN_REQUIRED=40300 main.m

I placed this in the clang_user_options variable in my .vimrc and code completion worked.

let g:clang_user_options='-fblocks -isysroot /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator4.3.sdk -D__IPHONE_OS_VERSION_MIN_REQUIRED=40300'

Then I added a function to vim to run xcodebuild. This function runs xcodebuild and puts the output from the run into the quickfix list. From there you can navigate the build errors by using :cwindow. Call the function from the vim command line like :call XCodeBuild

function! XCodeBuild()
  let l:command = 'xcodebuild -sdk iphonesimulator4.3'
  let l:out = system(l:command)
  cexpr l:out
endfunction
15Apr/112

Exception unsafe

I was reading up about exception safe code in C++ and the difference between the strong and basic exception guarantees. The article here (http://www.gotw.ca/gotw/082.htm) outlines what it means for code to satisfy the basic exception safety guarantee, however it doesn’t give an example. So I came up with something that is exception unsafe. The code below doesn’t satisfy the basic guarantee because it fails to delete the memory allocated to the pointer.

Exception unsafe.

#include <iostream>
#include <stdexcept>

using namespace std;

void h()
{
  throw logic_error((string("Failure in ") + string(__func__)));
}

void g()
{
  int* pi = new int;
  h();
  delete pi;
}

void f()
{
  try
  {
    g();
  }
  catch (const exception& e)
  {
    cout << e.what() << endl;
  }
}

int main(int argc, char** argv)
{
  f();
}

To ensure that it meets the basic guarantee the exception needs to be caught in g() and the memory deallocated. The code below does this. Unfortunately there is a bit of awkwardness here: note the double delete. One is in the bad path where the exception is caught and the other is in the good code path. It would be nice if the double delete wasn’t necessary. Other languages have a finally statement that is executed in both the good and bad case, however C\++ doesn’t have this but the problem is still solvable as shown below.

Basic exception safety met.

#include <iostream>
#include <stdexcept>

using namespace std;

void h()
{
  throw logic_error((string("Bad logic in ") + string(__func__)));
}

void g()
{
  int* pi = new int;
  try
  {
    h();
  }
  catch (const exception& e)
  {
    cout << "Caught exception: \"" << e.what() << "\" in " << __func__  << endl;
    delete pi;
    throw;
  }
  delete pi;
}

void f()
{
  try
  {
    g();
  }
  catch (const exception& e)
  {
    cout << "Caught exception: \"" << e.what() << "\" in " << __func__  << endl;
  }
}

int main(int argc, char** argv)
{
  f();
}

The lack of a finally statement in C++ isn’t a problem as long as the principle Resource Acquisition is Initialization is applied. It’s fairly straightforward and with a little bit of thought I think that most good programmers would realize what to do here in order to avoid having a double delete. The trick is to write a wrapper class to manage the resource and in the destructor clean up any allocated memory. If you follow this guideline then there is no need for C++ to have a finally statement. This is mentioned in the the C++ FAQ in section 17.6. The following code applies RAII. I’ve found myself doing this in code I’ve writted without explicitly realizing the technique I was applying had a name assoiciated with it.

Basic exception safety with RAII.

#include <iostream>
#include <stdexcept>

using namespace std;

class Resource
{
  private:
    int* pi;

  public:
    Resource()
    {
      cout << __func__ << endl;
      pi = new int;
    }

    ~Resource()
    {
      cout << __func__ << endl;
      delete pi;
    }
};

void h()
{
  throw logic_error((string("Bad logic in ") + string(__func__)));
}

void g()
{
  Resource r;
  h();
}

void f()
{
  try
  {
    g();
  }
  catch (const exception& e)
  {
    cout << "Caught exception: \"" << e.what() << "\" in " << __func__  << endl;
  }
}

int main(int argc, char** argv)
{
  f();
}
28Mar/110

Java JDKS

I’ve been doing some Java development recently mainly remedial work. I’ve also learnt some lessons about what Java promises and fails to deliver. Anyway the build system we’re using is Maven and it was building things fine on one of my development machines. However I tried to build the project on another host and got this error:

ERROR] execute error
java.lang.IllegalStateException: no source files
        at com.sun.tools.javac.main.Main.error(Main.java:162)
        at com.sun.tools.javac.main.Main.compile(Main.java:347)
        at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:132)
        at com.mysema.maven.apt.AbstractProcessorMojo.execute(AbstractProcessorMojo.java:193)
        at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
        at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
        at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
        at org.apache.maven.lifecycle.DefaultLifecycleExecutor.forkProjectLifecycle(DefaultLifecycleExecutor.java:1205)
        at org.apache.maven.lifecycle.DefaultLifecycleExecutor.forkLifecycle(DefaultLifecycleExecutor.java:1038)
        at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:643)
        at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
        at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
        at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
        at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
        at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
        at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
        at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
        at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
        at org.codehaus.classworlds.Launcher.main(Launcher.java:375)

This had me puzzled for hours. I cleaned the tree and did a rebuild but still got the same error. I rm -rf’ed the tree, got the latest from source control, did a rebuild but still got the same error. Then I had thought maybe it’s the different JDK’s installed. The machine that the build worked on had the Oracle JDK installed whereas the machine that I got this exception on had the OpenJDK installed. But aren’t the JDK’s meant to be compatible? Isn’t that what Java promises - the ability to run anywhere and it will output the same shit? I installed the Oracle JDK and reran the build and to my amusement it worked validating my cynicism about Java. So much for Java’s platform independence - it can’t even do the same thing on the same platform.

Filed under: java, programming No Comments
6Jan/110

Using SSH to proxy HTTP traffic

This is a neat way to setup a HTTP proxy. It lets me route HTTP traffic from my local machine at home through another virtual machine elsewhere on the internet that I administer before it makes it to the target url. On my home machine I just typed the following:

ssh -f -N -D 8888 name@your-virtual-host

This will tell ssh to listen on port 8888 on your local machine. Any connections to that port will be passed to the machine your-virtual-host. To verify that ssh is listening on that port type

$netstat -tn
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State
tcp        0      0 192.168.1.10:46488          208.111.39.234:22           ESTABLISHED
tcp        0      0 127.0.0.1:8888              127.0.0.1:41789             ESTABLISHED
tcp        0      0 192.168.1.10:43547          192.168.1.102:3389          ESTABLISHED
tcp        0      0 192.168.1.10:46486          208.111.39.234:22           ESTABLISHED

Then in my browser, in my case Firefox, I configured the proxy connection settings to use the SOCKS proxy at localhost and port 8888. Now every website I visit will see the connection originating from the virtual machine and not from the machine my browser is running on.

Filed under: note to self, ssh No Comments
18Dec/100

Notes about Git

These are my notes about git for use as quick reference. The notes describe using git and git with svn. In the case of svn the header is marked with svn.


Fundamentals

Typically a version control system has a repository where it stores the history of all changes. Then there’s your local development directory where you checkout a copy of the repositories contents, do some edits and then commit them back to the repository. Git breaks this model and adds an intermediate stage called the index that sits between your working directory and the git repository. Changes make there way from the working tree to the index then to the repository.

Commit Workflow

Figure 1. Commit Workflow

Checking out repository
Use git clone ... to clone a git repository locally.

git clone ssh://git@github.com/repo_path/repo.git

Checking out repository (svn)
You can use git with a svn repository. Note that this step can take a few hours because git checkouts out every revision in the svn repository and commits it locally.

git svn clone http://llvm.org/svn/llvm-project/libcxx/trunk libcxx

Updating your sources

git pull

Updating your sources (svn)

git svn rebase

Updating your sources after editing

sashan@cyclops clang master $ git svn rebase
include/clang/AST/Stmt.h: needs update
lib/AST/Stmt.cpp: needs update
update-index --refresh: command returned error: 1

Oops … git’s complaining that you have edited files in the working tree. Use git stash to save the changes locally.

sashan@cyclops clang master 1 $ git stash
Saved working directory and index state WIP on master: 3923c27 Fix diagnostic reporting of previous default statements.
HEAD is now at 3923c27 Fix diagnostic reporting of previous default statements.

Now you can rebase.

sashan@cyclops clang master $ git svn rebase
        M       docs/UsersManual.html
r121768 = 024394010048e704192f4f34f21836a550f37a9f (refs/remotes/git-svn)

And then apply the stashed changes

sashan@cyclops clang master 1 $ git stash pop

Committing changes in the working tree directly to the repository
This will add all editted files to index and then to repository. It will also prompt you to enter a commit log message. The -a parameter tells git to add all edited files.

sashan@cyclops clang master $ git commit -a

Selecting the files to commit
This stages files for committing adding them to the index.

sashan@cyclops clang master $ git add <file name>

After this you should run git commit to commit the files. This will commit the changes to your local repository.

Committing to a svn repository
When you’re happy with the changes made locally you can commit them to the upstream repository as shown below:

sashan@cyclops clang master $ git svn dcommit

Generating a patch for email submission
This will generate a patch file of the most recent commit that you can email as an attachment.

sashan@cyclops clang master $ git format-patch -1 -p
0001-Fix-diagnostic-reporting-of-previous-default-stateme.patch

Adding edited files to the index
Sometimes you want to add changes in the working tree to the index without committing to the repository at the same time. In this case doing a git commit -a won’t help. You need to use git add. The example below shows a session using git add -ui. The -u is for update and means that only modified paths in the working tree will be added to the index. New paths won’t. The -i is for interactive and therefore git will prompt you for a response. In the session below I enter interactive mode, choose to update the index, and then select a range of paths, 1 to 7, to update the index with. Importantly, after finishing the update I hit enter to exit the interactive update mode.

sashan@cyclops clang use_std_set_for_switch_cases $ git add -ui
           staged     unstaged path
  1:    unchanged      +51/-31 include/clang/AST/Stmt.h
  2:    unchanged        +6/-1 lib/AST/Stmt.cpp
  3:    unchanged        +4/-4 lib/Sema/JumpDiagnostics.cpp
  4:    unchanged        +5/-3 lib/Sema/SemaCodeComplete.cpp
  5:    unchanged       +14/-6 lib/Sema/SemaStmt.cpp
  6:    unchanged        +8/-2 lib/Serialization/ASTReaderStmt.cpp
  7:    unchanged        +3/-3 lib/Serialization/ASTWriterStmt.cpp

*** Commands ***
  1: status       2: update       3: revert       4: add untracked
  5: patch        6: diff         7: quit         8: help
What now> 2
           staged     unstaged path
  1:    unchanged      +51/-31 include/clang/AST/Stmt.h
  2:    unchanged        +6/-1 lib/AST/Stmt.cpp
  3:    unchanged        +4/-4 lib/Sema/JumpDiagnostics.cpp
  4:    unchanged        +5/-3 lib/Sema/SemaCodeComplete.cpp
  5:    unchanged       +14/-6 lib/Sema/SemaStmt.cpp
  6:    unchanged        +8/-2 lib/Serialization/ASTReaderStmt.cpp
  7:    unchanged        +3/-3 lib/Serialization/ASTWriterStmt.cpp
Update>> 1-7
           staged     unstaged path
* 1:    unchanged      +51/-31 include/clang/AST/Stmt.h
* 2:    unchanged        +6/-1 lib/AST/Stmt.cpp
* 3:    unchanged        +4/-4 lib/Sema/JumpDiagnostics.cpp
* 4:    unchanged        +5/-3 lib/Sema/SemaCodeComplete.cpp
* 5:    unchanged       +14/-6 lib/Sema/SemaStmt.cpp
* 6:    unchanged        +8/-2 lib/Serialization/ASTReaderStmt.cpp
* 7:    unchanged        +3/-3 lib/Serialization/ASTWriterStmt.cpp
Update>>
updated 7 paths

*** Commands ***
  1: status       2: update       3: revert       4: add untracked
  5: patch        6: diff         7: quit         8: help
What now> 7
Bye.

Bisecting to find the commit that introduced new behaviour
git bisect lets you find a commit that resulted in different behaviour. The man page describes it as git-bisect - Find by binary search the change that introduced a bug but it’s a slightly misleading since it really finds the commit that introduced a certain behaviour and behaviour in this case, doesn’t have to be a bug.

Anyway, to start, find out the commit hashes within which you know the behaviour changed and pass them to `git start … ` as shown below with the oldest commit hash first:

sashan@cyclops src (no $ git bisect start 22e6ee96f2daab9b7fcc135eff8ec133579da12d 7639a06e1d7f014929a9ab22fe5a35153bf9be85

git will checkout a branch with the head set to a commit in the middle of the 2 commits identified by the hashes above. Build the program and test for the desired behaviour. If what you see in the new build is the desired behaviour then mark it with git bisect good otherwise mark it with git bisect bad. Keep doing this until there are 0 bisections left. The head of the tree now will be at the commit that introduced the change.

23Nov/100

Investigating Asynchronous Workflows in F#

This article examines 4 common basic paradigms for implementing request response behaviour using F#. These are the 1) single-threaded synchronous model, 2) multi-threaded synchronous model 3) single-threaded asynchronous model 4) multi-threaded asynchronous model. We use the Net.WebRequest class to exercise the various I/O models. The source code for the article is here


Single-threaded synchronous model

let request_url (url : string) =
  let tid = string(Threading.Thread.CurrentThread.ManagedThreadId)
  Console.WriteLine("Thread id = " + tid + ", Request for " + url)
  let wr = Net.WebRequest.Create(url)
  use response = wr.GetResponse()
  let tid = string(Threading.Thread.CurrentThread.ManagedThreadId)
  Console.WriteLine("Thread id = " + tid + ", Response for " + url)

///run the single threaded synchronous calls
let ss_run urls =
  Console.WriteLine("Single-threaded synchronous requests ...")
  List.iter request_url urls

The fundamental properties to note here are that it’s single-threaded and that the responses to a request happen in sequence. Below is the example output when this code is run.

Single-threaded synchronous requests ...
Thread id = 1, Request for http://www.google.com
Thread id = 1, Response for http://www.google.com
Thread id = 1, Request for http://www.microsoft.com
Thread id = 1, Response for http://www.microsoft.com
Thread id = 1, Request for http://www.yahoo.com
Thread id = 1, Response for http://www.yahoo.com
Thread id = 1, Request for http://www.wordpress.com
Thread id = 1, Response for http://www.wordpress.com
Thread id = 1, Request for http://www.blizzard.com
Thread id = 1, Response for http://www.blizzard.com
Thread id = 1, Request for http://www.valvesoftware.com
Thread id = 1, Response for http://www.valvesoftware.com

Multi-threaded synchronous model

In this model multiple threads handle the initiation of the request. The request is made synchronously and the thread is blocked while waiting for the response.

let request_url (url : string) =
  let tid = string(Threading.Thread.CurrentThread.ManagedThreadId)
  Console.WriteLine("Thread id = " + tid + ", Request for " + url)
  let wr = Net.WebRequest.Create(url)
  use response = wr.GetResponse()
  let tid = string(Threading.Thread.CurrentThread.ManagedThreadId)
  Console.WriteLine("Thread id = " + tid + ", Response for " + url)

let create_thread request url =
  let helper request url =
    let tid = string(Threading.Thread.CurrentThread.ManagedThreadId)
    request url

  let t = new Threading.Thread(Threading.ThreadStart(fun _ -> helper request url))
  t

///run the multi-threaded synchronous version
let ms_run urls =
  Console.WriteLine("Multi-threaded synchronous requests ...")
  let tasks = List.map (fun url -> create_thread request_url url) urls
  List.iter (fun (thread : Threading.Thread) -> thread.Start()) tasks
  //wait for all threads to complete
  List.iter (fun (thread : Threading.Thread) -> thread.Join()) tasks

Single-threaded asynchronous model

Here I attempted to implement a single-threaded process to perform request response behaviour asynchronously and ended up bashing my head up against the problem. Eventually I gave up and came to the conclusion that one can’t do this easily under .NET. Apparently you have to invoke some magic using a SynchronizationContext class mixed in with some other .NET voodoo to do it and even then I’m not sure if that would work.

Am I being overly pedantic? No, I don’t think so since single-threaded programs performing asynchronous I/O have been in use for longer than .NET has been in existence. In fact, programs that behave in this fashion are currently in use in network appliances running Linux in various companies in all parts of the world. A use case for .NET needing to support this model is outlined. Imagine you were given the task of porting some code written on Linux platform where this model was employed onto Windows. Typically that code would be using select or libevent to multiplex input from various sources into a single thread. The safest way to port it would be to keep using the same single-threaded asynchronous model and not introduce a multi-threaded framework for the sake of implementing asynchronous i/o. Other reasons are listed below:

  • more threads increases the amount of time the OS spends context switching thus degrading performance
  • multi-threaded programs are harder to write correctly because for one your program is now non-deterministic.
  • If .NET supported this model then they wouldn’t have to advise having to do silly things like switching the context back to the UI thread so that you can update a variable owned by that thread.
  • It’s clean and elegant and aesthetically pleasing.

The example program below was my attempt to put together a single-threaded program that makes multiple requests and handles the response in the same thread. However, the output shows that the response comes back in a different thread from the original. The program also uses asynchronous workflows. This is a neat language feature that no other .NET language currently supports and will be explained in the final section.

let st_request (url : string) max_urls (count : int ref)
  (finished : Threading.ManualResetEvent) =
  async {
    let tid = string(Threading.Thread.CurrentThread.ManagedThreadId)
    let wr = Net.WebRequest.Create url
    Console.WriteLine("Thread id = " + tid + ", Request for " + url)
    use! response = wr.AsyncGetResponse()
    count := !count + 1
    let tid = string(Threading.Thread.CurrentThread.ManagedThreadId)
    Console.WriteLine("Thread id = " + tid + ", Response for " + url)
    if !count = 6 then
      finished.Set() |> ignore
    ()
  }

///run the single threaded asynchronous version
let sa_run =
  let finished = new Threading.ManualResetEvent(false)
  let count = ref 0
  (fun (urls : list<string>) ->
    Console.WriteLine("Single-threaded asynchronous requests ...")
    List.iter (fun url ->
      Async.StartImmediate(st_request url urls.Length count finished)) urls (*** <2> ***)
    finished.WaitOne() |> ignore)

The output from running this code is shown below:

Single-threaded asynchronous requests ...
Thread id = 1, Request for http://www.google.com
Thread id = 1, Request for http://www.microsoft.com
Thread id = 1, Request for http://www.yahoo.com
Thread id = 1, Request for http://www.wordpress.com
Thread id = 1, Request for http://www.blizzard.com
Thread id = 1, Request for http://www.valvesoftware.com
Thread id = 13, Response for http://www.microsoft.com
Thread id = 13, Response for http://www.yahoo.com
Thread id = 11, Response for http://www.google.com
Thread id = 14, Response for http://www.valvesoftware.com
Thread id = 10, Response for http://www.blizzard.com
Thread id = 13, Response for http://www.wordpress.com

It’s clearly not single-threaded.


Multi-threaded asynchronous model

Like the multi-threaded synchronous model a thread is spawned for each request. The important thing to note here is that, because the program is now multi-threaded, we have to take care of shared variables. In the sample below the count reference is shared between multiple threads therefore increments to it must be atomic. There is nothing inherently special about an async workflow that makes it auto-magically thread safe.

In other languages handling the response from a request is often the responsibility of a callback function. It’s named callback because it is called back when the response is received and ready for processing. F# brings a sanitized model for dealing with asynchronous i/o through the use of asynchronous workflows to the table. These get rid of the control flow inversion [async_model] associated with the callback style of programming. This is a big thing in .NET world because no other .NET language can do this although OCaml has had support for this for a while via lwt Anyone that has had to work with programs that have complicated interactions with each other via message passing will see the value in what F# does here.

let mt_request (url : string) max_urls (count : int ref)
  (finished : Threading.ManualResetEvent) =
  async {
    let tid = string(Threading.Thread.CurrentThread.ManagedThreadId)
    let wr = Net.WebRequest.Create url
    Console.WriteLine("Thread id = " + tid + ", Request for " + url)
    use! response = wr.AsyncGetResponse() (*** <1> ***)
    Threading.Interlocked.Increment(count) |> ignore (*** <2> ***)
    let tid = string(Threading.Thread.CurrentThread.ManagedThreadId)
    Console.WriteLine("Thread id = " + tid + ", Response for " + url)
    if !count = 6 then
      finished.Set() |> ignore
    ()
  }

let ma_run =
  let count = ref 0
  let finished = new Threading.ManualResetEvent(false)
  (fun (urls : list<string>) ->
    Console.WriteLine("Multi-threaded asynchronous requests ... ")
    List.iter (fun url ->
      Async.Start(mt_request url urls.Length count finished)) urls (*** <3> ***)
    finished.WaitOne() |> ignore)
  1. This line is the heart and soul of the async workflow and there’s quite a bit going on this single line. There’s the use of the keyword use, a ! and a call to AsyncGetResponse. The use keyword indicates that the resource bound to the name response implements the IDisposable interface. Anything that implements IDisposable needs to call Dispose to release resources allocated to it back to the operating system. Binding it with use indicates we want the the lifetime of the resource tied to the lexical scope of the name, effectively freeing the programmer from the need to manually call Dispose since this will be automatically handled for us when the name response goes out of scope. The ! operator here tells it to 1) initiate the request and 2) wait without blocking the current thread for the response. Note that there is no registering of a callback function to handle the response. Control flow is not inverted, and variables that were in scope prior to the request being sent are still in scope when the response arrives. These properties are not maintained in a language that requires the use of a callback function to handle the response.
  2. We use Interlocked.Increment to increment the count. This is atomic and therefore thread safe.
  3. Async.Start will spawn a new thread for each async workflow.

Output from running code that uses the functions above is shown below:

Multi-threaded asynchronous requests ...
Thread id = 17, Request for http://www.google.com
Thread id = 20, Request for http://www.blizzard.com
Thread id = 17, Request for http://www.valvesoftware.com
Thread id = 21, Request for http://www.wordpress.com
Thread id = 19, Request for http://www.yahoo.com
Thread id = 18, Request for http://www.microsoft.com
Thread id = 14, Response for http://www.microsoft.com
Thread id = 16, Response for http://www.google.com
Thread id = 16, Response for http://www.yahoo.com
Thread id = 10, Response for http://www.valvesoftware.com
Thread id = 14, Response for http://www.blizzard.com
Thread id = 13, Response for http://www.wordpress.com

Note the thread hopping that occurs. The response is not guaranteed to be in the same thread as the request was made from.


Conclusion

In summary we’ve shown how to implement various common models using F# for request response style programming. Unfortunately it’s not straightforward to implement single-threaded asynchronous code in .NET because it keeps wanting to create threads behind your back. Regardless the neat thing about F# and asynchronous workflows is that they get rid of the flow of control inversion associated with callback style programming resulting in other useful properties such as exception propagation and resource lifetime management via lexical scoping being maintained.


References

Filed under: Uncategorized No Comments