The initial few weeks of the project were spent researching related work, to see how others had used gaze. If you are looking for recent academic papers relating to gaze interaction in computing, the COGAIN Association website is an excellent place to start. They even provide a section dedicated to gaze controlled games.
Various tools were also looked into, in order to see which would be the most appropriate for the project. Two game engines were looked at, Torque 3D and the Unity 3D Engine. Torque does receive much praise, is reasonably priced and is cited in at least one paper investigating gaze in FPS. However rather than a trial version of the engine, Torque provide only a demo, so I was unwilling to commit to purchasing it. Unity do provide a trial version of their software and is very impressive, you could undoubtedly have a game up and running in a very short time using this application. However it is a very high abstracted tool so it is not so clear how easy it might be to integrate the engine with the Tobii SDK or indeed voice recognition. So yet again I was unwilling to commit to this engine.
OGRE (Object-Oriented Graphics Rendering Engine) was also closely examined. It is a scene-oriented, flexible 3D engine, as opposed to a game engine. Ogre is written in C++ and is designed to make it easier and more intuitive to produce applications utilising hardware-accelerated 3D graphics. It is an excellent open source graphics engine with a very active community and was shown to work with both voice recognition and eye tracking by Wilcox et al in 2008. The wealth of resources, active community and proven record of having worked with voice and gaze data in the past seemed to make it the ideal tool to develop the gaze/voice game in.
Then at the start of June Jon Ward of Acuity-Ets made the T60 Tobii Eye Tracker and Tobii SDK available to us. The documentation which came along with the Tobii SDK is very good and makes it very easy to get started working with it. The Tobii Eye Tracker Components API, or TetComp, is an interface containing everything you need to programmatically get real-time high-precision gaze data from Tobii eye tracker hardware. It is a type library implemented as a set of COM objects, making it accessible to many modern high-level program languages for Microsoft 32-bit platforms.
With this in mind I decided to try a quick experiment and try and get the Tobii T60 to work within XNA. XNA is a set of tools with a managed runtime environment provided by Microsoft that facilitates computer game development. The XNA framework is based on the native implementation of the .Net Compact framework 2.0. I had done a lot of work using XNA during the year and have found it very user-friendly allowing for very rapid development of games. I felt that my familiarity with it would eliminate much of the extra time I would need to learn OGRE, so if I could integrate it with Tobii COM API, it would make it the ideal candidate in which to develop my game.
The TetComp interface was indeed easily referenced from within my XNA application and I was quickly able to get the application to start calibrating the eye tracker and gather gaze data from within XNA. Now all that was required was to integrate voice recognition. Having already installed Microsoft Speech SDK, I found that I was able to easily reference it's COM API. So now I had access to both gaze and voice data from within a development framework I was familiar with. I could start on the game proper.
Various tools were also looked into, in order to see which would be the most appropriate for the project. Two game engines were looked at, Torque 3D and the Unity 3D Engine. Torque does receive much praise, is reasonably priced and is cited in at least one paper investigating gaze in FPS. However rather than a trial version of the engine, Torque provide only a demo, so I was unwilling to commit to purchasing it. Unity do provide a trial version of their software and is very impressive, you could undoubtedly have a game up and running in a very short time using this application. However it is a very high abstracted tool so it is not so clear how easy it might be to integrate the engine with the Tobii SDK or indeed voice recognition. So yet again I was unwilling to commit to this engine.
OGRE (Object-Oriented Graphics Rendering Engine) was also closely examined. It is a scene-oriented, flexible 3D engine, as opposed to a game engine. Ogre is written in C++ and is designed to make it easier and more intuitive to produce applications utilising hardware-accelerated 3D graphics. It is an excellent open source graphics engine with a very active community and was shown to work with both voice recognition and eye tracking by Wilcox et al in 2008. The wealth of resources, active community and proven record of having worked with voice and gaze data in the past seemed to make it the ideal tool to develop the gaze/voice game in.
Then at the start of June Jon Ward of Acuity-Ets made the T60 Tobii Eye Tracker and Tobii SDK available to us. The documentation which came along with the Tobii SDK is very good and makes it very easy to get started working with it. The Tobii Eye Tracker Components API, or TetComp, is an interface containing everything you need to programmatically get real-time high-precision gaze data from Tobii eye tracker hardware. It is a type library implemented as a set of COM objects, making it accessible to many modern high-level program languages for Microsoft 32-bit platforms.
With this in mind I decided to try a quick experiment and try and get the Tobii T60 to work within XNA. XNA is a set of tools with a managed runtime environment provided by Microsoft that facilitates computer game development. The XNA framework is based on the native implementation of the .Net Compact framework 2.0. I had done a lot of work using XNA during the year and have found it very user-friendly allowing for very rapid development of games. I felt that my familiarity with it would eliminate much of the extra time I would need to learn OGRE, so if I could integrate it with Tobii COM API, it would make it the ideal candidate in which to develop my game.
The TetComp interface was indeed easily referenced from within my XNA application and I was quickly able to get the application to start calibrating the eye tracker and gather gaze data from within XNA. Now all that was required was to integrate voice recognition. Having already installed Microsoft Speech SDK, I found that I was able to easily reference it's COM API. So now I had access to both gaze and voice data from within a development framework I was familiar with. I could start on the game proper.
Above is a video of my initial experimentation within XNA showing the tracking of gaze data and use of voice recognition. Its not too fancy but is proof enough that eye tracker works well within the XNA framework.
No comments:
Post a Comment