Majumder Aditi - Introduction to visual computing: core concepts in computer vision, graphics, and image processing

In the context of visual computing, data can be thought of as a function that depends on one or more independent variables. For example, audio can be thought of as one dimensional (1D) data that is dependent on the variable time. Thus, it can be represented as A ( t ) where t denotes time. An image is data that is two dimensional (2D) data dependent on two spatial coordinates x and y and can be denoted as I ( x , y ) . A video is three dimensional (3D) data that is dependent on three variables two spatial coordinates ( x , y ) and one temporal coordinate t . It can therefore be denoted by V ( x , y , t ) .

The simplest visualization of a multidimensional data is a traditional plot of the dependent variable with respect to the independent ones, as illustrated in ).

Figure 1.1 Most common visualization of 1 D (left) and 2 D (right) data. The 1 D data shows the population of US ( Y axis) during the 20th century (specified by time in the Xaxis) while the 2 D data shows the surface elevation ( Z axis) of a geographical region (specified by X and Y axes). This is often called height field .

Figure 1.2 Conducive Visualizations: An image is represented as three 2 D functions, R ( x , y ) , G ( x , y ) and B ( x , y ) . But instead of three height fields, a more conducive visualization is where every pixel ( x , y ) is shown in RGB color (left). Similarly, volume data T ( x , y , z ) is visualized by depicting the data at every 3 D point by its transparency (right).

Data exists in nature as a continuous function. For example, the sound we hear changes continuously over time; the dynamic scenes that we see around us also change continuously with time and space. However, if we have to digitally represent this data, we need to change the continuous function to a discrete one, i.e. a function that is only defined at certain values of the independent variable. This process is called discretization . For example, when we discretize an image defined in continuous spatial coordinates ( x , y ) , the values of the corresponding discrete function are only defined at integer locations of ( x , y ) , i.e. pixels.

Figure 1.3 This figure illustrates the process of sampling. On top left, the function f ( t ) (curve in blue) is sampled uniformly. The samples are shown with red dots and the values of t at which the function is sampled is shown by the vertical blue dotted lines. On top right, the same function is sampled at double the density. The corresponding discrete function is shown in the bottom left. On the bottom right, the same function is now sampled nonuniformly i.e. the interval between different values of t at which it is sampled varies.

A sample is a value (or a set of values) of a continuous function f ( t ) at a specified value of the independent variable t . Sampling is a process by which one or more samples are extracted from a continuous signal f ( t ) thereby reducing it to a discrete function f ^ ( t ) . The samples can be extracted at equal intervals of the independent variable. This is termed as uniform sampling. Note that the density of sampling can be changed by changing the interval at which the function is sampled. If the samples are extracted at unequal intervals, then it is termed as non uniform sampling. These are illustrated in .

The process of getting the continuous function f ( t ) back from the discrete function f ^ ( t ) is called reconstruction . In order to get an accurate reconstruction, it is important to sample f ( t ) adequately during discretization. For example, in , a high frequency sine wave (in blue) is sampled in two different ways, both uniformly, shown by the red and blue samples. But in both cases the sampling frequency or rate is not adequate. Hence, a different frequency sine wave is reconstructed for blue samples a zero frequency sine wave and for red samples a much lower frequency sine wave than the original wave. These incorrectly reconstructed functions are called aliases (for imposters) and the phenomenon is called aliasing .

Figure 1.4 This figure illustrates the effect of sampling frequency on reconstruction. Consider the high frequency sine wave shown in blue. Consider two types of sampling shown by the blue and red samples respectively. Note that none of these sample the high frequency sine wave adequately and hence the samples represent sine waves of different frequencies.

This brings us to the question of what is adequate sampling frequency ? As it turns out, for sine or cosine waves of frequency f , one has to sample them at a minimum of double the frequency, i.e. 2 f , to assure correct reconstruction. This rate is called the Nyquist sampling rate . However, note that the reconstruction is not a process of merely connecting the samples. The reconstruction process is discussed in details in later chapters.

We just discussed adequate sampling for sine and cosine waves. But, what is adequate sampling for a general signal not a sine or a cosine wave ? To answer this question, we have to turn to the operation complementary to reconstruction, called decomposition . Legendary 19th century mathematician, Fourier, showed that any periodic function f ( t ) can be decomposed into a number of sine and cosine waves which when added together give the function back. We will revisit Fourier decomposition in greater detail at has to be sampled at least at a rate of 6 f to assure a correct reconstruction.

Figure 1.5 This figure illustrates how addition of different frequency sine waves results in the process of generation of general periodic signals.

A analog or continuous signal can have any value of infinite precision. However, whenever it is converted to digital signal, it can only have a limited set of value. So a range of analog signal values is assigned to one digital value. This process is called quantization . The difference between the original value of a signal and its digital value is called the quantization error .

The discrete values can be placed at equal intervals resulting in uniform step size in the range of continuous values. Each continuous value is usually assigned the nearest discrete value. Hence, the maximum error is half the step size. This is illustrated in .