Statistical significance in the era of big data Cover Image

Istotność statystyczna w czasach big data
Statistical significance in the era of big data

Author(s): Mirosław Szreder
Subject(s): Economy
Published by: Główny Urząd Statystyczny
Keywords: statistical inference; hypothesis testing; statistical significance; p-value; big data; Bayesian approach

Summary/Abstract: The development of new technologies has affected both the procedures of traditional statistical surveys and the perception of their results in the light of other available sources of information. In this connection, the role of the verification of statistical hypotheses and of the interpretation and presentation of its results, including the use of statistical significance and p-value, has recently returned as a frequent topic for discussion among the scientific community. The author was inspired to write this paper by a wave of discussion regarding this matter held at the beginning of 2019 in the "Nature" and "The American Statistician" journals. The aim of the paper is to present the opportunities provided and challenges posed by the use of big data to the hypothesis verification process and to statistical inference, both in the traditional and Bayesian approaches. The author explains the necessity of discontinuing adopting excessive simplifications while performing statistical inference and presenting the results of the verification of hypotheses. This involves both the postulate to pay greater attention to the quality of sampling data, especially in the case of data originating from big data sets, as well as the postulate to provide full information about the statistical model on the basis of which the inference is being performed.

  • Issue Year: 64/2019
  • Issue No: 11
  • Page Range: 42-57
  • Page Count: 16
  • Language: Polish
Toggle Accessibility Mode