Nobuoki Eshima - Statistical Data Analysis and Entropy

Statistical Data Analysis and Entropy — read online for free the complete book (whole text) full work

Below is the text of the book, divided by pages. System saving the place of the last page read, allows you to conveniently read the book "Statistical Data Analysis and Entropy" online for free, without having to search again every time where you left off. Put a bookmark, and you can go to the page where you finished reading at any time.

Light

Font size:

↓

↑

Georgia Tahoma Arial Verdana

Reset

Interval:

↓

↑

Bookmark:

Make

1234567...123

Springer Nature Singapore Pte Ltd. 2020

N. Eshima Statistical Data Analysis and Entropy Behaviormetrics: Quantitative Approaches to Human Behavior https://doi.org/10.1007/978-981-15-2552-0_1

1. Entropy and Basic Statistics

Nobuoki Eshima

(1)

Center for Educational Outreach and Admissions, Kyoto University, Kyoto, Japan

Nobuoki Eshima

Email:

1.1 Introduction

Entropy is a physical concept to measure the complexity or uncertainty of systems under studies, and it is used in thermodynamics, statistical mechanics, and information theory, and so on; however, the definitions are different in the research domains. In measuring information of events, entropy was introduced to study communication systems by Shannon [ treats the information of continuous variables, and t and F statistics are expressed through entropy.

1.2 Information

Let be a sample space; be an event; and let the probability of event A. The information of event A is defined mathematically according to the probability, not the content itself. Smaller the probability of an event is, greater we feel its value. Based on our intuition, the mathematical definition of information [] is given by

Definition 1.1

For , the information of A is defined by

(1.1)

where

In which follows, the base of the logarithm is e and the notation () is simply denoted by

In this case, the unit is called nat, i.e., natural unit of information. If , i.e., event A always occurs, then and it implies that event A has no information. The information measure has the following properties:

1. (i)
   For events A and B, if , the following inequality holds:

   

   (1.2)
2. (ii)
   If events A and B are statistically independent, then, it follows that

   

   (1.3)

Proof

Inequality () is trivial. In (ii), we have

(1.4)

From this,

Example 1.1

In a trial drawing a card from a deck of cards, let events A and B be ace and heart, respectively. Then,

Since , corresponding to (), we have

and the events are statistically independent, so we also have Eq. ().

Remark 1.1

In information theory, base 2 is usually used for logarithm. Then, the unit of information is referred to as bit. One bit is the information of an event with probability

1.3 Loss of Information

When we forget the last two columns of a phone number with 10 digits, e.g., 075-753-25##, then there is nothing except that the right number can be restored with probability , if there is no information about the number. In this case, the loss of the information about the right phone number is In general, the loss of the information is defined as follows:

Definition 1.2

Let A and B be events such that Then, the loss of information concerning event A by using or knowing event B for A is defined by

Next page

1234567...123

Light

Font size:

↓

↑

Georgia Tahoma Arial Verdana

Reset

Interval:

↓

↑

Bookmark:

Make

Similar books «Statistical Data Analysis and Entropy»

Look at similar books to Statistical Data Analysis and Entropy. We have selected literature similar in name and meaning in the hope of providing readers with more options to find new, interesting, not yet read works.

Simona Balzano

Statistical Learning and Modeling in Data Analysis: Methods and Applications

Kohl

Introduction to statistical data analysis with R

Phil Attard

Entropy Beyond the Second Law: Thermodynamics and Statistical Mechanics for Equilibrium, Non-equilibrium, Classical, and Quantum Systems

Martins Luiz Felipe

Mastering Python data analysis become an expert at using Python for advanced statistical analysis of data using real-world examples

Chakraborty Soubhik

Computational Musicology in Hindustani Music

Richard M. Heiberger

Statistical Analysis and Data Display

Ton J. Cleophas

Statistical Analysis of Clinical Data on a Pocket Calculator

James Gentle

Statistical Analysis of Financial Data: With Examples in R

Siegmund Brandt

Data Analysis: Statistical and Computational Methods for Scientists and Engineers

Roger D. Peng

Exploratory data analysis with R

Bruce Ratner

Statistical and Machine-Learning Data Mining: Techniques for Better Predictive Modeling and Analysis of Big Data, Second Edition

Rae R. Newton

Your Statistical Consultant: Answers to Your Data Analysis Questions

Discussion, reviews of the book Statistical Data Analysis and Entropy and just readers' own opinions. Leave your comments, write what you think about the work, its meaning or the main characters. Specify what exactly you liked and what you didn't like, and why you think so.