{"id":39,"date":"2011-07-13T03:36:43","date_gmt":"2011-07-13T03:36:43","guid":{"rendered":"http:\/\/constructive-noise.info\/?page_id=39"},"modified":"2016-09-21T13:20:36","modified_gmt":"2016-09-21T17:20:36","slug":"sear-rl","status":"publish","type":"page","link":"http:\/\/constructive-noise.info\/?page_id=39","title":{"rendered":"SEAR-RL"},"content":{"rendered":"<p>SEAR-RL stand for &#8216;See Hear &#8211; Real Life&#8217; it is a continuation of a project I started long ago.<\/p>\n<p>AKA: \u00a0Probably my last hurrah in technology.<\/p>\n<p>AKA: \u00a0Seeing with &#8220;Blobs of Sound&#8221; &#8220;Sight with Sound&#8221; &#8220;Video to Audio Translation&#8221;<\/p>\n<p><iframe loading=\"lazy\" title=\"SEAR-RL A Kinect hack for 3D Translation into Audio.  AKA Seeing with sound.\" width=\"630\" height=\"354\" src=\"https:\/\/www.youtube.com\/embed\/HDpIluRSFbk?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: center;\"><a href=\"https:\/\/i0.wp.com\/constructive-noise.info\/wordpress\/wp-content\/uploads\/2011\/07\/Helmet-Kinect-1.jpg\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" class=\"size-medium wp-image-41\" title=\"Helmet-Kinect-1\" src=\"https:\/\/i0.wp.com\/constructive-noise.info\/wordpress\/wp-content\/uploads\/2011\/07\/Helmet-Kinect-1.jpg?resize=300%2C168\" alt=\"Hard Hat with Kinect on it.\" width=\"300\" height=\"168\" srcset=\"https:\/\/i0.wp.com\/constructive-noise.info\/wordpress\/wp-content\/uploads\/2011\/07\/Helmet-Kinect-1.jpg?resize=300%2C168 300w, https:\/\/i0.wp.com\/constructive-noise.info\/wordpress\/wp-content\/uploads\/2011\/07\/Helmet-Kinect-1.jpg?resize=1024%2C576 1024w, https:\/\/i0.wp.com\/constructive-noise.info\/wordpress\/wp-content\/uploads\/2011\/07\/Helmet-Kinect-1.jpg?w=1280 1280w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p><span class=\"Apple-style-span\" style=\"color: #000000; font-size: 17px; line-height: 25px;\">Background:<\/span><\/p>\n<p>A little over a decade over when I was finishing up college, I got the question in my head of what a audio-only video game would be like. \u00a0I was taking a multimedia class at the time, so as my project for the class, toyed around with a couple of game mechanics to see if it could be done.<\/p>\n<p>After college I somehow fell in with working with a couple of accessibility focused companies,\u00a0<a title=\"Orcca Technology\" href=\"http:\/\/www.orcca.com\/\" target=\"_blank\">Orcca Technology<\/a>\u00a0and <a title=\"All-In-Play\" href=\"http:\/\/allinplay.com\/\" target=\"_blank\">All-In-Play Games<\/a>. \u00a0The nice thing was it\u00a0kept my interest in the accessibility and how to create an audio-only game in my head.<\/p>\n<p>Now you have to understand, audio-only games are not a new idea. \u00a0 There is a whole subculture to them, as you can check out here <a title=\"Audiogame.net\" href=\"http:\/\/www.audiogames.net\/\" target=\"_blank\">audiogames.net<\/a>. \u00a0But most of the games I tried out never really seemed to solve the problem of translating graphics\u00a0adequately for me.<\/p>\n<p>So the question evolved into how who one go about translating graphics into sound so someone who is visually impaired could play something like Doom and Unreal.<\/p>\n<p>There have been attempts in the past, but the solution always seemed to be what I call the &#8216;Tagging&#8217; issue. \u00a0 Objects could be tagged in a game to tell the player what and where they are, but the tag do not always represent the volume or description of the object. \u00a0Also an\u00a0environment\u00a0needs tagging\u00a0before a visually impaired player can effectively play the game. \u00a0What is one supposed to do if one cannot always\u00a0guarantee\u00a0a tagging pass?<\/p>\n<p>Eventually, I went to graduate school at the <a title=\"ETC at CMU\" href=\"http:\/\/www.etc.cmu.edu\/\" target=\"_blank\">ETC at Carnegie Mellon<\/a>\u00a0in 2005, and I spent a semester in an independent study trying to prototype a simple idea how one might go about getting around \u00a0that problem of translating \u00a0a 3D\u00a0environment into sound.<\/p>\n<p>I would say the project was generally a success. \u00a0 Yes, there were a couple of issues, but in \u00a0translating the visuals\u00a0on-screen into a useful sound-scape, my advisor and I thought there was\u00a0something\u00a0there.<\/p>\n<p>However, it was while working on the project, I realized if one could feed in 3D data realtime to the system, one could\u00a0translate the real world into audio which people could use to navigate the world.<\/p>\n<p>I wanted to expand the idea further, but 3D scanners were pretty expensive and not so portable at the time.\u00a0So I graduated and moved\u00a0to other things.<\/p>\n<p>Then last year Microsoft released the <a title=\"Kinect Homepage\" href=\"http:\/\/www.xbox.com\/en-US\/kinect\" target=\"_blank\">Kinect<\/a>, and more recently the official Windows <a title=\"Kinect SDK\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/redmond\/projects\/kinectsdk\/\" target=\"_blank\">SDK<\/a>. \u00a0I realized with a little work, the Kinect could be a portable 3D Scanner.<\/p>\n<p>So I dug out my old project and merged the two together, you can see the results in the video.<\/p>\n<p>Now there have been similar systems in the past like the <a title=\"vIOCe System\" href=\"http:\/\/www.seeingwithsound.com\/\" target=\"_blank\">vIOCe<\/a>\u00a0and more recently the <a title=\"NAVI\" href=\"http:\/\/hci.uni-konstanz.de\/blog\/2011\/03\/15\/navi\/?lang=en\" target=\"_blank\">NAVI<\/a> system\u00a0\u00a0from University of Konstanz.<\/p>\n<p>Where vIOCe uses 2D static images I am trying to use 3D real world data. \u00a0 NAVI uses 3D data but uses haptics and tags to communicate data about objects in the real world. \u00a0With SEAR I am trying to use only sound to give impressions of volume of objects in the real world.<\/p>\n<p>Anyway, this little personal project has been a long time in the making. \u00a0I figure I might as well share my results.<\/p>\n<p>Hope you enjoy.<\/p>\n<p>-Stan<\/p>\n","protected":false},"excerpt":{"rendered":"<p>SEAR-RL stand for &#8216;See Hear &#8211; Real Life&#8217; it is a continuation of a project I started long ago. AKA: \u00a0Probably my last hurrah in technology. AKA: \u00a0Seeing with &#8220;Blobs of Sound&#8221; &#8220;Sight with Sound&#8221; &#8220;Video to Audio Translation&#8221; &nbsp; Background: A little over a decade over when I was finishing up college, I got [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":3,"comment_status":"open","ping_status":"closed","template":"","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"class_list":["post-39","page","type-page","status-publish","hentry"],"jetpack_shortlink":"https:\/\/wp.me\/P1wOQH-D","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"http:\/\/constructive-noise.info\/index.php?rest_route=\/wp\/v2\/pages\/39","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/constructive-noise.info\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/constructive-noise.info\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/constructive-noise.info\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/constructive-noise.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=39"}],"version-history":[{"count":0,"href":"http:\/\/constructive-noise.info\/index.php?rest_route=\/wp\/v2\/pages\/39\/revisions"}],"wp:attachment":[{"href":"http:\/\/constructive-noise.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=39"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}