Network and Data Communication cha3-4

Chapter 03Basics of AudioB i f A diOutline3.1 Digitization of Sound31i i i i f S d3.2 MIDI: Musical Instrument Digital32MIDI:Musical Instrument DigitalInterface3.1Digitization of Sound3.1 Digitization of SoundWhat is Sound?Sound is a wave phenomenon like light, but is macroscopic and involves molecules of air being compressed and expanded under the action ofsome physical devicesome physical device.(a) For example, a speaker in an audio system vibrates back and forth andproduces a longitudinal pressure wave that we perceive as sound.(b) Since sound is a pressure wave, it takes on continuous values, as opposed todigitized ones.(c) Even though such pressure waves are longitudinal, they still have ordinarywave properties and behaviors, such as reflection (bouncing), refraction(g g g y)(change of angle when entering a medium with a different density) anddiffraction (bending around an obstacle).(d) If we wish to use a digital version of sound waves we must form digitizedprepresentations of audio information.gDigitizationDigitization means conversion to a stream of numbers, and preferablyh b h ld b i f ffi ithese numbers should be integers for efficiency.Fig. 3.1 shows the 1-dimensional nature of sound: amplitude values p,depend on a 1D variable, time.3.1The graph in Fig. 3.1 has to be made digital in both time and amplitude. To digitize, the signal must be sampled in each dimension: in time, and in amplitude.(a) Sampling means measuring the quantity we are interested in, usually at evenly-spaced intervals.(b) The first kind of sampling, using measurements only at evenly spaced time(b)The first kind of sampling using measurements only at evenly spaced timeintervals, is simply called, sampling. The rate at which it is performed is called the sampling frequency (see Fig. 3.2(a)).(c) For audio, typical sampling rates are from 8 kHz (8,000 samples per second) to(c)For audio typical sampling rates are from8kHz(8000samples per second)to48 kHz. This range is determined by Nyquist theorem discussed later.(d) Sampling in the amplitude or voltage dimension is called quantization. Fig.3.2(b) shows this kind of sampling.32(b)h thi ki d f li3.2Thus to decide how to digitize audio data we need to answer the following questions:1. What is the sampling rate?1What is the sampling rate?2. How finely is the data to be quantized, and is quantization uniform?3. How is audio data formatted? (file format)()3.2 MIDI: Musical Instrument3.MI I Musical Instrument Digital InterfaceUse the sound card's defaults for sounds: → use a simple scripting language and hardware setup called MIDI.MIDI Overview(a) MIDI is a scripting language ㅡit codes “events" that stand for the productionof sounds. e.g., a MIDI event might include values for the pitch of a single note,f d MIDI t i ht i l d l f th it h f i l tits duration, and its volume.(b) MIDI is a standard adopted by the electronic music industry for controllingdevices, such as synthesizers and sound cards, that produce music.d i h h i d d d h d i(c) The MIDI standard is supported by most synthesizers, so sounds created on onesynthesizer can be played and manipulated on another synthesizer and soundreasonably close.(d) Computers must have a special MIDI interface, but this is incorporated intomost sound cards. The sound card must also have both D/A and A/D converters.pMIDI ConceptsMIDI channels are used to separate messages.(a) There are 16 channels numbered from 0 to 15. The channel forms the last 4bits (the least significant bits) of the message.(b) Usually a channel is associated with a particular instrument: e.g., channel 1 (b)Usually a channel is associated with a particular instrument:e g channel1is the piano, channel 10 is the drums, etc.(c) Nevertheless, one can switch instruments midstream, if desired, andassociate another instrument with any channel.i t th i t t ith h lSystem messages(a) Several other types of messages, e.g. a general message for all instrumentsindicating a change in tuning or timing.indicating a change in t ning or timing(b) If the first 4 bits are all 1s, then the message is interpreted as a system commonmessage.The way a synthetic musical instrument responds to a MIDI message is usually by simply ignoring any play sound message message is usually by simply ignoring any play sound messagethat is not for its channel.-If several messages are for its channel, then the instrument responds, provided it is g,p,p multi-voice, i.e., can play more than a single note at once.i f h i i h h i bIt is easy to confuse the term voice with the term timbre ㅡthe latter is MIDI terminology for just what instrument that is t i t b l t d i d t i li it i trying to be emulated, e.g. a piano as opposed to a violin: it is the quality of the sound.(a) An instrument (or sound card) that is multi-timbral is one that is capable of(a)An instrument(or sound card)that is multi timbral is one that is capable ofplaying many different sounds at the same time, e.g., piano, brass, drums, etc.(b) On the other hand, the term voice, while sometimes used by musicians to meanthe same thing as timbre, is used in MIDI to mean every different timbre andpitch that the tone module can produce at the same time.Different timbres are produced digitally by using a patch ㅡthe set of control settings that define a particular timbre.the set of control settings that define a particular timbrePatches are often organized into databases, called banks.General MIDI: A standard mapping specifying what instruments (what patches) will be associated with what channels.h l(a) In General MIDI, channel 10 is reserved for percussion instruments, and(a)In General MIDI,channel10is reserved for percussion instruments,andthere are 128 patches associated with standard instruments.(b) For most instruments, a typical message might be a Note On message(meaning, e.g., a keypress and release), consisting of what channel, what(meaning e g a keypress and release)consisting of what channel whatpitch, and what " velocity" (i.e., volume).(c) For percussion instruments, however, the pitch data means which kind ofdrum.drum(d) A Note On message consists of " status" byte which channel, what pitch ㅡfollowed by two data bytes. It is followed by a Note O message, which also has a pitch (which note to turn o) and a velocity (often set to zero).has a pitch(which note to turn o)and a velocity(often set to zero)Hardware Aspects of MIDI Hardware Aspects of MIDI3.33.3The physical MIDI ports consist of 5-pin connectors for IN and OUT, as well as a third connector called THRU.(a) MIDI communication is half-duplex.(b) MIDI IN is the connector via which the device receives all MIDI data.(c) MIDI OUT is the connector through which the device transmits all the MIDIdata it generates itself.(d) MIDI THRU is the connector by which the device echoes the data it receivesfrom MIDI IN. Note that it is only the MIDI IN data that is echoed by MIDI THRU -all the data generated by the device itself is sent via MIDI OUT.3.43.434Chapter 04 Fundamental Concepts F d t l C t in Videoin VideoOutline4.1 Analog Video41Analog Video4.2 Digital Video4.1 Analog VideoAn analog signal f(t) samples a time-varying image. So-called “progressive"scanning traces through a complete picture (a frame)row g“progressive" scanning traces through a complete picture (a frame) row-wise for each time interval. In TV, and in some monitors and multimedia standards as well, another system, called “interlaced" scanning is used:a) The odd-numbered lines are traced rst, and then the even-numbered lines are )traced. This results in “odd" and “even" fields -two fields make up one frame.b) In fact, the odd lines (starting from 1) end up at the middle of a line at the end of the odd field, and the even scan starts at a half-way point.,y pc) Figure 4.1 shows the scheme used. First the solid (odd) lines are traced, P to Q, then R to S, etc., ending at T; then the even field starts at U and ends at V.d)The jump from Q to R etc in Figure 41is called the horizontal retrace during d) The jump from Q to R, etc. in Figure 4.1 is called the horizontal retrace, during which the electronic beam in the CRT is blanked. The jump from T to U or V to P is called the vertical retrace.4.1Because of interlacing, the odd and even lines are displaced in time from each other | generally not noticeable except when very fast action is taking place on screen, when blurring may occur.place on screen when blurring may occurFor example, in the video in Fig. 4.2, the moving helicopter is blurred more than is the still background.4.24.2 Digital Video4.2Digital VideoThe advantages of digital representation for video are many The advantages of digital representation for video are many.For example:(a) Video can be stored on digital devices or in memory, ready to be processed(noise removal, cut and paste, etc.), and integrated to various multimediaapplications;(b) Direct access is possible, which makes nonlinear video editing achievable as asimple, rather than a complex, task;(c) Repeated recording does not degrade image quality;()p g g g q y;(d) Ease of encryption and better tolerance to channel noise.Chroma SubsamplingSince humans see color with much less spatial resolution than they see black and ,gwhite, it makes sense to “decimate" the chrominance signal.To begin with, numbers are given stating how many pixel values, per four original pixels, are actually sent:The chroma scheme\4:4:4"indicates that no chroma is(a)The chroma subsampling scheme \4:4:4 indicates that no chroma subsampling isused: each pixel's Y, Cb and Cr values are transmitted, 4 for each of Y, Cb, Cr.(b)The scheme \4:2:2" indicates horizontal subsampling of the Cb, Cr signals by afactor of2That is of four pixels horizontally labelled as0to3all four Ys arefactor of 2. That is, of four pixels horizontally labelled as 0 to 3, all four Ys aresent, and every two Cb's and two Cr's are sent, as (Cb0, Y0)(Cr0,Y1)(Cb2,Y2)(Cr2, Y3)(Cb4, Y4), and so on (or averaging is used).The scheme\4:1:1"subsamples horizontally by a factor of4(c)The scheme \4:1:1" subsamples horizontally by a factor of 4.(d)The scheme \4:2:0" subsamples in both the horizontal and vertical dimensions bya factor of 2. Theoretically, an average chroma pixel is positioned between therows and columns as shown Fig.4.3.d l h Fi434.3HDTV (High Definition TV)The main thrust of HDTV (High Definition TV) is not to (g )increase the “definition" in each unit area, but rather to increase the visual field especially in its width.(a) The first generation of HDTV was based on an analog technology developed by Sony and NHK in Japan in the late 1970s.(b) MUSE (Multiple sub-Nyquist Sampling Encoding) was an improved NHK HDTV with hybrid analog/digital technologies that was put in use in the 1990s. It has 1,125 scan lines, interlaced (60 fields per second), and 16:9 aspect ratio.,,(p ),p(c) Since uncompressed HDTV will easily demand more than 20 MHz bandwidth, which will not t in the current 6 MHz or 8 MHz channels, various compression techniques are being investigated.q g g(d) It is also anticipated that high quality HDTV signals will be transmitted using more than one channel even after compression.A brief history of HDTV evolution:A brief history of HDTV evolution:(a) In 1987, the FCC decided that HDTV standards must be compatible with theexisting NTSC standard and be conned to the existing VHF (Very HighFrequency) and UHF (Ultra High Frequency) bands.(),y,,p(b) In 1990, the FCC announced a very different initiative, i.e., its preference for afull-resolution HDTV, and it was decided that HDTV would be simultaneously broadcast with the existing NTSC TV and eventually replace it.()g p p g,y(c) Witnessing a boom of proposals for digital HDTV, the FCC made a key decisionto go all-digital in 1993. A “grand alliance" was formed that included four main proposals, by General Instruments, MIT, Zenith, and AT&T, and by Thomson, pPhilips, Sarno and others.(d) This eventually led to the formation of the ATSC (Advanced TelevisionSystems Committee) | responsible for the standard for TV broadcasting ofHDTV.(e) In 1995 the U.S. FCC Advisory Committee on Advanced Television Servicerecommended that the ATSC Digital Television Standard be adopted.。