Silverlight 5 P-Invoke: Using Kinect SDK from Silverlight

Away from all the discussions about whether Silverlight is dead or not, The Silverlight 5 RC contains the previously announced P-Invoke feature which enables you to call Win32 style APIs from a trusted Silverlight application.

There are some attempts to use Kinect from Silverlight (here) but this was before MS released the official SDK. we will use Silverlight 5 P-Invoke feature to call the Kinect SDK APIs.

P-Invoke in Silverlight works just like P-Invoke on the desktop. you use the DllImport attribute to import the APIs and you declare in your code the dependent types (Enums, structs, etc.)

1. [DllImport("MSRKINECTNUI.DLL")]
2.       private static extern HRESULT NuiInitialize(uint dwFlags);

I tried to keep the classes and methods in library identical to the ones in the official SDK. I implemented only a couple of the available APIs (with the help of the Coding4Fun Kinect Toolkit and Reflector).

To test the library we will create a simple application:

- Create a new Silverlight 5 application

- Add a reference to the KinectSilverlightLibrary.

- Change the properties of the project: enable running out of browser.

- Change the Out-of-Browser settings to require elevated trust

- In the App.xaml.cs file, define a new property of type Runtime, this property represents the Kinect runtime, in the Application_Startup event handler initialize the runtime and in the Application_Exit event handler shut down the runtime

1. Runtime kinectRuntime = new Runtime();
2. public Runtime KinectRuntime
3. {
4.     get
5.     { return kinectRuntime; }
6. }
7. private void Application_Startup(object sender, StartupEventArgs e)
8. {
9.     this.RootVisual = new MainPage();
10.     kinectRuntime.Initialize(RuntimeOptions.UseColor | RuntimeOptions.UseDepth );
11. }
12. private void Application_Exit(object sender, EventArgs e)
13. {
14.     kinectRuntime.Uninitialize();
15. }

- In the MainPage.xaml file add two image controls that we will use to display the video and depth streams coming from Kinect

1. <Grid x:Name="LayoutRoot" Background="White" Loaded="LayoutRoot_Loaded">
2.         <Grid.ColumnDefinitions>
3.             <ColumnDefinition/>
4.             <ColumnDefinition/>
5.         </Grid.ColumnDefinitions>
6.         <Image Name="image1" Stretch="Fill" Grid.Column="0" />
7.         <Image Name="image2" Stretch="Fill" Grid.Column="1" />
8.     </Grid>

- In the MainPage.xaml.cs add the following lines of code

1. private void LayoutRoot_Loaded(object sender, RoutedEventArgs e)
2.         {
3.            Runtime runtime = (App.Current as App).KinectRuntime;
4.            runtime.VideoStream.OpenStream(ImageType.Color, ImageResolution.Resolution640x480);
5.           runtime.DepthStream.OpenStream(ImageType.Depth, ImageResolution.Resolution320x240);
6.          runtime.VideoFrameReady += new EventHandler<ImageFrameReadyEventArgs>(runtime_VideoFrameReady);
7.            runtime.DepthFrameReady += new EventHandler<ImageFrameReadyEventArgs>(runtime_DepthFrameReady);
8.         }
9.     
10.         void runtime_VideoFrameReady(object sender, ImageFrameReadyEventArgs e)
11.         {
12.             Dispatcher.BeginInvoke(() => {
13.                 image1.Source = e.ImageFrame.ToBitmapSource();
14.             });
15.         }
16.         void runtime_DepthFrameReady(object sender, ImageFrameReadyEventArgs e)
17.         {
18.             Dispatcher.BeginInvoke(() =>
19.             {
20.                 image2.Source = e.ImageFrame.ToBitmapSource();
21.             });
22.  
23.         }

When the page is loaded we open the video and depth streams and subscribe to the VideoFrameReady and DepthFrameReady events, in the event handler we retrieve the ImageFrame and convert it to a bitmap (I used the WritableBitmapEx library)

Here’s the application running

 

You are free to continue adding the rest of the APIs to this library.

You can download the source code from here: