Invited Paper:
High Definition 3D auditory displays and microphone arrays for the use with future 3D TV
SUZUKI Yoiti(*1,*2), TREVINO Jorge(*1,*2), OKAMOTO Takuma(*3), CUI Zhenglie(*1),
SAKAMOTO Shuichi(*1,*2), and IWAYA Yukio(*4)
(*1) Research Institute of Electrical Communication, Tohoku University, Japan(*2) Graduate School of Information Sciences, Tohoku University
(*3) National Institute of Information and Communications Technology
(*4) Faculty of Engineering, Tohoku Gakuin University
Abstract :
Sound information generally has a strong and positive synergism with visual information.
Therefore, development of high-definition 3D auditory displays as well as sound acquisition
systems must be an important issue to realize future high-definition 3D communications and
broadcasting. Moreover, to realize future communications and broadcasting with a high sense-of-presence,
it is important to recall that we humans are active creatures, moving through the environment
to acquire accurate spatial information. For instance, in terms of spatial hearing, such movement
is known to be effective for improving the precision of auditory spatial recognition. We designate
this style of listening as active listening. Therefore, it is particularly important that auditory
displays to reproduce/synthesize 3D sound fields be responsive to a listener’s movement, at least
to a listener’s head rotation. Auditory displays matching the motions of active listening are therefore
eagerly sought for use in future communications and broadcasting, where high-definition 3D TV will be
widely used.
In this presentation, we first introduce a high-definition auditory display based on high-order
Ambisonics (HOA) architecture with the fifth order: the highest order realized in 2011. This system is
implemented with a surrounding loudspeaker array of 157 loudspeakers. These systems are expected to be
useful to realize high-definition 3D auditory displays providing high sense-of-presence. We then
introduce an audio-visual 3D displays consisting of the above mentioned HOA 3D auditory display and a 3D
projection display on the market. This system has a very low temporal disparity between audio and visual
signals and thus is expected to be useful as experimental systems to accumulate new knowledge related to
human perceptions, which is crucially important for the advancement of 3D communications and broadcasting
systems.
Secondly, we introduce our high-definition small spherical microphone array based on SENZI (Symmetrical
object with ENchased Zillion microphones) architecture and its implementation with 252 channels of
microphones. SENZI can sense spatial sound information comprehensively and precisely so that the sensed
(recorded) sounds can be suitably reproduced by auditory displays that are responsive to a listener’s
movement. As proper reproduction of the signal sensed by SENZI is possible at any place for any listener
and at any time after the sensing (recording), high-definition 3D sound field reproduction can be
provided with SENZI, beyond place and time.
Parts of this research are supported by Tohoku University GCOE program CERIES, Grants-in-Aid nos.19001004
& 24240016 to SY from MEXT and JSPS, and by SCOPE (no. 082102005) to Sakamoto from MIC Japan.
Biography :
Yoiti Suzuki graduated from the School of Engineering, Tohoku Univeristy, Sendai, Japan, in 1976,
where he also received M.Sci. and Ph.D. degrees in electrical and communication engineering, in,
respectively, 1978 and 1981. He is currently a Professor, Acoustic Information Systems Laboratory in
Human Information Systems Division, Research Institute of Electrical Communication, Tohoku University,
Sendai, Japan. His standpoint is that human beings are regarded as the extreme source and recipient of
information in any communication systems. With this standpoint, he has been devoting into developing
advanced 3D acoustic communication systems based on good knowledge of human auditory system as well as
multimodal perception relating to hearing.
He served as a President of the Acoustical Society of Japan from 2005 to 2007. He was a recipient of the
Takenaka and RCA David Sarnoff Sholarships and has received several awards from the Awaya Kiyoshi Award
and the Sato Prize from the Acoustical Society of Japan, the Funai Best Paper Award from FIT2005. He is a
Fellow of the Acoustical Society of America, Virtual Rreality Society of Japan and IEICE, Japan and is an
International Honorary Member of the Acoustical Society of Korea. He is serving as a member of the
Information and Communications Council, MIC, since 2009, the Chairperson of the ITU Committee, MIC, since
2011 and the Chairperson of URCF (Ultra Realistic Communications Forum) since 2012.
Light Field Camera and IP display
Toru Iwane
Advanced Digital Imaging Laboratory, Nikon Corporation, JapanAbstract :
Recently a light-field camera has become common and popular since Lytro’s one was made public. This “light-field” camera is likely to be known limitedly as a “refocusing camera”; a camera with which you can change focus conditions of the images after you take picture. This light field camera is also useful for acquiring 3D image and is especially well-suited for an IP type 3D display because a light field camera detects 3D light conditions of scene. We have developed a light field camera and a 3D image surface display compatible with the camera. This Display is characterizes that it reconstructs not a real 3D image in space but 3D image surface. And the resolution is not limited by micro-lens number because detector plane is not conjugated with image plane and can be regarded as another domain. I report this camera-display system and logical approach to 3D display from light field camera theorem
Biography :
Toru Iwane has been in charge of developing and researching novel optical devices for imaging in NIKON
CORPORATION after studying Astrophysics in Kyoto Univ.
He developed automatic focusing system and focus-tracking system for camera. Subsequently invented and
realized “vari-bright” viewfinder display using polymer network liquid crystal And now he is engaged in
investigating especially computational photography, including light field camera and integral display.
Standardization and service prospect of 3DTV broadcasting
Jisang Yoo
Kwangwoon University, KoreaAbstract :
The recent emergence of multimedia applications has motivated a considerable
interest in the development of three-dimensional (3D) TV aimed at providing
as a real dynamic scene. Ideal scene reconstruction can be achieved by
holograms, in principle. They allow any number of viewers simultaneously,
and provide for each of them the stereoscopic depth cue, the lens
accommodation cue, and the motion parallax cue. However, the current state
of technology does not allow real-time holographic video acquisition and
display systems.
Therefore currently the most promising candidates are stereo and multi-view
systems based on stereo vision. The human brain can process subtle
difference between the images that are presented to the left and right eyes
to perceive a three-dimensional real world.
In this talk, how TV service will be evolved in the future will be briefly
introduced. Then, 3D activities going on in Korea will be introduced
including the activities ARMI, next generation broadcast standardization
forum (NGBSF) and 3DTV research group of the Korean society of broadcast
engineers. Almost for 20 years, many companies in Korea have been developing
the 3D related products and some of them are already in the market. And
finally in 2010, the high definition 3DTV broadcasting by terrestrial was
started first time in the world and last year we started the HD 3DTV
broadcasting service in the national wide commercial terrestrial channel. In
this talk, the service prospect of 3DTV broadcasting and standardization
work for it will be introduced.
Biography :
Jisang Yoo received the B.S. and M.S. degrees from Seoul National University, Seoul, Korea in 1985 and 1987, all in electronics engineering, and Ph.D. degree from Purdue University, West Lafayette, IN, USA, in electrical engineering in 1993, respectively. From September 1993 to August 1994, he worked as a senior research engineer in industrial electronics R&D center at Hyundai Electronics Industries Co., Ltd, ichon, Korea, in the area of image compression and HDTV. From 1994 to 1997, he was an assistant professor with the department of electronics engineering, Hallym University, Chuncheon, Korea. He is currently a professor with the department of electronics engineering, Kwangwoon University, Seoul, Korea. His research interests are in 3D image processing, nonlinear digital filtering, and computer vision. Now, he is the chair of the promotion committee of HD 3DTV Broadcasting service and also the chair of 3DTV project group of TTA in Korea.
Development of Basic Technologies for Creative 3D Contents and 3D Display Systems
Jar-Ferr Yang and Hung-Ming Wang
Department of Electronical Engineering, Institute of Computer and Communication Engeneering, National Cheng Kung University, TaiwanAbstract :
To promote the 3D developments and related industries, we are currently running a project named“Development of Basic Technologies for Creative 3D Contents and 3D Display Systems”. In this invited paper, we will briefly overview the project, which is composed of three important subprojects including comfortable 3D video display engine technologies, 3D video cloud GPU realization technologies, and 3D capture and content production technologies. Currently, we have implemented many CPU/GPU-based kernel techniques such as high performance stereo matching, versatile multiview computation, super-resolution interpolation, and web-based 3D player. With the accomplishment of this project, we expect it in the near future will benefit3D developments and facilitate 3D creative applications in the world.
Biography :
Jar-Ferr Yang received his BS, MS, and Ph.D. degrees from Chung-Yuan Christian University, Taiwan and
National Taiwan University, and University of Minnesota, Minneapolis, USA in, 1977, 1979, and 1988,
respectively. In 1988, he joined the National Cheng Kung University (NCKU) as associate professor and
promoted to full and distinguished professors in 1994 and 2004, respectively. He was the Chairperson of
Graduate Institute of Computer and Communication Engineering and Director of the Electrical and
Information Technology Center. Currently, he is the Director of Center of Tomorrow Ubiquitous Computing
and Hypermedia (TOUCH) Services.
During 2004 - 2005, he was Distinguished Lecturer of the IEEE Circuits and Systems Society. During 2008 -
2009, he was the Chair of the IEEE CAS Multimedia Systems and Applications Technical Committee during 2008
– 2009 and tthe Chairman of IEEE Tainan Section during 2009-2010. He was Associate Editor of IEEE
Transaction on Circuits and Systems for Video Technology and IEEE Circuits and Devices Magazine.
Currently, he is an Associate Editor of EURASIP Journal of Advances in Signal Processing and an
Editorial Board Member of IET Signal Processing. He received the NSC Excellent Research Award in 2008
and the Outstanding Electrical Engineering Professor Award in 2010, Taiwan. He has published over 104
journal and 167 conference papers. His research areas include multimedia processing and coding, and
their applications in smart living and learning system integrations. He is a Fellow of IEEE for his
contributions to fast algorithms and efficient realization of video and audio coding.