Chris Bore - The Art of DSP: An innovative introduction to DSP

The Art of DSP

common knowledge in the art

Claude Shannon: Communication in the Presence of Noise

(Commenting on his Sampling Theorem)

About this book

"The only really valuable thing is intuition."

Albert Einstein

This book is about DSP Digital Signal Processing.

DSP used to be a specialist subject but now most engineers and scientists and many in other fields such as finance will use it at some time. Signals are used everywhere and almost all processing is now digital, so knowing about DSP is essential.

DSP is often presented mathematically as an academic subject: but in practice you are not usually asked to derive or prove the mathematical tools, but to apply them. You need to understand what the tools do, their limitations, and how to use them.

DSP is an art: the expression of human creativity and imagination applied to design and to apply tools that shape the world in ways that are themselves works of art. The tools of this art are mathematics, analysis and programming.

This eBook focuses on The Art of DSP: learning the tools with understanding and intuition; using them with skill and ingenuity; applying them with imagination and creativity; and crafting with them in a careful, diligent way that is informed by practical reality.

This is not a text book or a reference. Nor does it teach programming or the maths of DSP. Its aim is to give you a basic and intuitive understanding that will inform your application of the tools, not to provide you with answers to exam questions.

This eBook written specifically for the Amazon Kindle is meant to be read as you would a novel: starting at the beginning, reading through in sequence, following the narrative. Not stopping to make notes, nor working through exercises or assignments, just reading it for enjoyment and to gain a first understanding of the subject.

To explore the content of this eBook in a more practical way you could register for the BORES on-line course The Practice of DSP which offers a course of study enlivened with practical and programming exercises, tests, and discussion forums. To extend the depth of your understanding you could buy some of the other BORES eBooks on DSP, that develop each topic in a more rigorous way, in depth.

A useful art

Given its reputation as an academic subject of dry mathematical rigour the idea of DSP as an art may seem odd at first. But the description is justified both by ancient usage and present reality.

The distinction between science for its own sake and its useful application was once expressed as science and the useful arts. Science studies the world: useful art shapes it. This is why an inventor in seeking to patent their work establishes prior art, and why the Constitution of the United States gives Congress a duty:

to promote the progress of science and useful arts

We still draw a similar distinction between academic study and application.

DSP is presented as an academic subject. Academic means relating to education and scholarship: cruelly we might say that academic implies that something is not of practical relevance - that theory is more important than practice. In DSP this comes through as too much focus on mathematics - especially on proofs and derivations - and too little on application and interpretation.

Applying DSP takes skill: the ability to do it well, which is gained through practice as well as expertise. Again referring to patent law, an invention must be described so that it could be made and used by:

a person skilled in the art

You can learn to drive: but to be a skilled driver you must develop an intuitive feel for what you are doing. You do not gain a skilled feel for DSP by learning the mathematics: you have to think around it, how it might be used, and about what it means.

Useful art is the application of human creative skill and imagination. Often art has a beauty or emotional power. DSP is an art. It takes skill and imagination to apply it well: and the tools as well as the products of their use often have great beauty and emotional power too. This book is about the art of DSP.

Its only a model

Professor Daphne L Jackson

(private remark to the author as a student)

DSP and mathematics

The foundations of DSP are mathematical but it is an applied subject. As a DSP engineer, you will rarely invent new mathematics. Most often, you will design new processing using existing mathematical tools and those tools will usually be readily available programming functions or hardware components. You dont need to understand the design of a hardware component in order to use it nor do you expect to examine the source code of a programming function in order to call it. Likewise, you do not need to understand the internal mathematics of a DSP process before you can apply it. You do need to know what the process does, how you can combine it with other processes, and how to use it.

Mathematics is not enough, either, for many applications. For instance in MPEG video compression we recognize that many blocks of video pixels are similar to data sent earlier, and exploit that by telling the receiver to re-use that data instead of re-sending it. That is an example of ingenuity of cleverness in solving a difficult problem as you might a puzzle. You probably wouldnt find that solution through mathematics it requires a problem-solving lateral-thinking mind to come up with such a thing. You can express it mathematically once you had thought of it, but that expression is awkward and artificial, whereas the idea is of breathtaking simplicity.

Abstraction

This is like abstraction in software development: you can use a software component or function without knowing why it works or how it does so, all you need to know is what it does and how to use it. At a lower level of abstraction someone must implement the function and they must know why it works and how: but abstraction lets us focus on using the function without worrying about its details. Mathematics in DSP is often the low level abstraction, and so we can use it without worrying about its proof or derivation or implementation. Knowing how the mathematics works and how it was derived is interesting and helpful background so it is worth exploring but it is not essential in itself. Even if you do know the detail you should not be thinking about it when you work at a higher level of abstraction. The abstraction lets you work more efficiently precisely because you dont carry all the low level baggage at all times: just as knowing how a car engine works probably helps you to be a more efficient driver but you dont want to drive with the hood up, looking at the engine.

Simplification

Most real signals cannot be described by simple equations with easy solutions so making simplifying approximations and understanding the limitations they introduce becomes important. Often the information is imperfect and so making reasonable assumptions becomes an art in itself. These real-world engineering problems require quite simple mathematics but a lot of quantitative thinking, reasoning, questioning, estimating and interpreting. These real-world aspects to DSP make it more interesting.

Very often in DSP the elegant and powerful mathematical models we apply do not quite fit with reality. The most famous theorem in DSP, Shannons Sampling Theorem, models a particular set of signals that behave in particular ways - it is just that those signals are very often not quite the signals you actually have. You can say that the Theorem is an approximation, or that it models signals that differ from what you might actually have, but you cannot complain that the real signals do not behave as they should - that they fail to conform to your simplified ideal model. Much of the mathematics that supports DSP - theorems and equations and formulae - is designed for an ideal Platonic world where things behave as they should but not as they do.