Download the full version and explore a variety of ebooks

or text books at https://ebookball.com

Natural Image Stattistics A Probabilistic Approach

to Early Computational vision 1st Edition by Aapo
Hyvarinen, Jarmo Hurri, Patrick O Hoyer ISBN
1848824904 9781848824904

_____ Follow the link below to get your download now _____

https://ebookball.com/product/natural-image-stattistics-a-
probabilistic-approach-to-early-computational-vision-1st-
edition-by-aapo-hyvarinen-jarmo-hurri-patrick-o-hoyer-
isbn-1848824904-9781848824904-14546/

Access ebookball.com now to download high-quality

ebooks or textbooks
Here are some recommended products for you. Click the link to
download, or explore more at ebookball.com

Natural Computing in Computational Finance Volume 4 1st

Edition by Anthony Brabazon, ‎Michael O Neill,
‎Dietmar Maringer ISBN 3642233368 9783642233364
https://ebookball.com/product/natural-computing-in-computational-
finance-volume-4-1st-edition-by-anthony-brabazon-aeurzmichael-o-neill-
aeurzdietmar-maringer-isbn-3642233368-9783642233364-13664/

Probabilistic Model for Structured Document Mapping 1st

Edition by Guillaume Wisniewski, Francis Maes, Ludovic
Denoyer, Patrick Gallinari 9783540734987
https://ebookball.com/product/probabilistic-model-for-structured-
document-mapping-1st-edition-by-guillaume-wisniewski-francis-maes-
ludovic-denoyer-patrick-gallinari-9783540734987-9670/

Computer Vision A Modern Approach 2nd Edition by David

Forsyth, Jean Ponce 013608592X 978-0136085928

https://ebookball.com/product/computer-vision-a-modern-approach-2nd-
edition-by-david-forsyth-jean-ponce-013608592x-978-0136085928-17226/

Algorithms for Image Processing and Computer Vision 2nd

Edition by Parker 0470643854 978-0470643853

https://ebookball.com/product/algorithms-for-image-processing-and-
computer-vision-2nd-edition-by-parker-0470643854-978-0470643853-17240/
Extraction of Semantic Dynamic Content from Videos with
Probabilistic Motion Models 1st edition by Gwenaelle
Piriou, Patrick Bouthemy, Jian Feng Yao ISBN 3540219828
9783540219828
https://ebookball.com/product/extraction-of-semantic-dynamic-content-
from-videos-with-probabilistic-motion-models-1st-edition-by-gwenaelle-
piriou-patrick-bouthemy-jian-feng-yao-
isbn-3540219828-9783540219828-9656/

My Lead Dog Was A Lesbian Mushing Across Alaska in the

Iditarod the World Most Grueling Race 1st Edition by Brian
Patrick O Donoghue ISBN 0679764119 9780679764113
https://ebookball.com/product/my-lead-dog-was-a-lesbian-mushing-
across-alaska-in-the-iditarod-the-world-most-grueling-race-1st-
edition-by-brian-patrick-o-donoghue-
isbn-0679764119-9780679764113-14992/

Speech and Language Processing An Introduction to Natural

Language Processing Computational Linguistics and Speech
Recognition 1st Edition by Daniel Saul Jurafsky, James
Martin 0130950696 9780130950697
https://ebookball.com/product/speech-and-language-processing-an-
introduction-to-natural-language-processing-computational-linguistics-
and-speech-recognition-1st-edition-by-daniel-saul-jurafsky-james-
martin-0130950696-9780130950697/

Fundamentals of Geriatric Medicine a case based approach

1st Edition by Rainier Patrick Soriano, Christine Cassel,
Rosanne Leipzig, Helen Fernadez ISBN 9780387323268
0387323260
https://ebookball.com/product/fundamentals-of-geriatric-medicine-a-
case-based-approach-1st-edition-by-rainier-patrick-soriano-christine-
cassel-rosanne-leipzig-helen-fernadez-
isbn-9780387323268-0387323260-308/

Fundamentals of Geriatric Medicine A Case Based Approach

1st Edition by Rainier Patrick Soriano, Christine K
Cassel, Rosanne Leipzig, Helen Fern ISBN 0387323260
9780387323268
https://ebookball.com/product/fundamentals-of-geriatric-medicine-a-
case-based-approach-1st-edition-by-rainier-patrick-soriano-christine-
k-cassel-rosanne-leipzig-helen-fern-
isbn-0387323260-9780387323268-2966/
Aapo Hyvärinen
Jarmo Hurri
Patrik O. Hoyer

Natural Image Statistics

A probabilistic approach to early
computational vision

December 11, 2008

Springer
Contents overview

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Part I Background
2 Linear filters and frequency analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3 Outline of the visual system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
4 Multivariate probability and statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
Part II Statistics of linear features
5 Principal components and whitening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6 Sparse coding and simple cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
7 Independent component analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
8 Information-theoretic interpretations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
Part III Nonlinear features & dependency of linear features
9 Energy correlation of linear features & normalization . . . . . . . . . . . . . . 209
10 Energy detectors and complex cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
11 Energy correlations and topographic organization . . . . . . . . . . . . . . . . . 249
12 Dependencies of energy detectors: Beyond V1 . . . . . . . . . . . . . . . . . . . . . 273
13 Overcomplete and non-negative models . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
14 Lateral interactions and feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
Part IV Time, colour and stereo
15 Colour and stereo images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
16 Temporal sequences of natural images . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
Part V Conclusion
17 Conclusion and future prospects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
Part VI Appendix: Supplementary mathematical tools
18 Optimization theory and algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
19 Crash course on linear algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
20 The discrete Fourier transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
21 Estimation of non-normalized statistical models . . . . . . . . . . . . . . . . . . . 437
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453

v
Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 What this book is all about . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 What is vision? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 The magic of your visual system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Importance of prior information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4.1 Ecological adaptation provides prior information . . . . . . . . . . 6
1.4.2 Generative models and latent quantities . . . . . . . . . . . . . . . . . . 8
1.4.3 Projection onto the retina loses information . . . . . . . . . . . . . . 9
1.4.4 Bayesian inference and priors . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.5 Natural images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5.1 The image space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.5.2 Definition of natural images . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.6 Redundancy and information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.6.1 Information theory and image coding . . . . . . . . . . . . . . . . . . . 13
1.6.2 Redundancy reduction and neural coding . . . . . . . . . . . . . . . . 15
1.7 Statistical modelling of the visual system . . . . . . . . . . . . . . . . . . . . . . . 16
1.7.1 Connecting information theory and Bayesian inference . . . . . 16
1.7.2 Normative vs. descriptive modelling of visual system . . . . . . 16
1.7.3 Towards predictive theoretical neuroscience . . . . . . . . . . . . . . 17
1.8 Features and statistical models of natural images . . . . . . . . . . . . . . . . 18
1.8.1 Image representations and features . . . . . . . . . . . . . . . . . . . . . . 18
1.8.2 Statistics of features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.8.3 From features to statistical models . . . . . . . . . . . . . . . . . . . . . . 19
1.9 The statistical-ecological approach recapitulated . . . . . . . . . . . . . . . . . 21
1.10 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Part I Background

2 Linear filters and frequency analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.1 Linear filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.1.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

vii
viii Contents

2.1.2 Impulse response and convolution . . . . . . . . . . . . . . . . . . . . . . 28

2.2 Frequency-based representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.2.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.2.2 Representation in one and two dimensions . . . . . . . . . . . . . . . 30
2.2.3 Frequency-based representation and linear filtering . . . . . . . . 33
2.2.4 Computation and mathematical details . . . . . . . . . . . . . . . . . . 37
2.3 Representation using linear basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.3.1 Basic idea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.3.2 Frequency-based representation as a basis . . . . . . . . . . . . . . . . 40
2.4 Space-frequency analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.4.2 Space-frequency analysis and Gabor filters . . . . . . . . . . . . . . . 43
2.4.3 Spatial localization vs. spectral accuracy . . . . . . . . . . . . . . . . . 46
2.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.6 Exercices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3 Outline of the visual system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

3.1 Neurons and firing rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.2 From the eye to the cortex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.3 Linear models of visual neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.3.1 Responses to visual stimulation . . . . . . . . . . . . . . . . . . . . . . . . 55
3.3.2 Simple cells and linear models . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.3.3 Gabor models and selectivities of simple cells . . . . . . . . . . . . 58
3.3.4 Frequency channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.4 Nonlinear models of visual neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.4.1 Nonlinearities in simple-cell responses . . . . . . . . . . . . . . . . . . 59
3.4.2 Complex cells and energy models . . . . . . . . . . . . . . . . . . . . . . 62
3.5 Interactions between visual neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.6 Topographic organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.7 Processing after the primary visual cortex . . . . . . . . . . . . . . . . . . . . . . 66
3.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.9 Exercices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

4 Multivariate probability and statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

4.1 Natural images patches as random vectors . . . . . . . . . . . . . . . . . . . . . . 69
4.2 Multivariate probability distributions . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.2.1 Notation and motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.2.2 Probability density function . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.3 Marginal and joint probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
4.4 Conditional probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.5 Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.6 Expectation and covariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.6.1 Expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.6.2 Variance and covariance in one dimension . . . . . . . . . . . . . . . 80
4.6.3 Covariance matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Contents ix

4.6.4 Independence and covariances . . . . . . . . . . . . . . . . . . . . . . . . . 81

4.7 Bayesian inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.7.1 Motivating example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
4.7.2 Bayes’ Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.7.3 Non-informative priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.7.4 Bayesian inference as an incremental learning process . . . . . 86
4.8 Parameter estimation and likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.8.1 Models, estimation, and samples . . . . . . . . . . . . . . . . . . . . . . . 88
4.8.2 Maximum likelihood and maximum a posteriori . . . . . . . . . . 89
4.8.3 Prior and large samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.10 Exercices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Part II Statistics of linear features

5 Principal components and whitening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

5.1 DC component or mean grey-scale value . . . . . . . . . . . . . . . . . . . . . . . 97
5.2 Principal component analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.2.1 A basic dependency of pixels in natural images . . . . . . . . . . . 99
5.2.2 Learning one feature by maximization of variance . . . . . . . . . 100
5.2.3 Learning many features by PCA . . . . . . . . . . . . . . . . . . . . . . . . 103
5.2.4 Computational implementation of PCA . . . . . . . . . . . . . . . . . . 104
5.2.5 The implications of translation-invariance . . . . . . . . . . . . . . . . 106
5.3 PCA as a preprocessing tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
5.3.1 Dimension reduction by PCA . . . . . . . . . . . . . . . . . . . . . . . . . . 108
5.3.2 Whitening by PCA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.3.3 Anti-aliasing by PCA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
5.4 Canonical preprocessing used in this book . . . . . . . . . . . . . . . . . . . . . . 113
5.5 Gaussianity as the basis for PCA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
5.5.1 The probability model related to PCA . . . . . . . . . . . . . . . . . . . 115
5.5.2 PCA as a generative model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
5.5.3 Image synthesis results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.6 Power spectrum of natural images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
5.6.1 The 1/ f Fourier amplitude or 1/ f 2 power spectrum . . . . . . . 116
5.6.2 Connection between power spectrum and covariances . . . . . . 119
5.6.3 Relative importance of amplitude and phase . . . . . . . . . . . . . . 120
5.7 Anisotropy in natural images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
5.8 Mathematics of principal component analysis* . . . . . . . . . . . . . . . . . . 122
5.8.1 Eigenvalue decomposition of the covariance matrix . . . . . . . . 123
5.8.2 Eigenvectors and translation-invariance . . . . . . . . . . . . . . . . . . 125
5.9 Decorrelation models of retina and LGN * . . . . . . . . . . . . . . . . . . . . . . 126
5.9.1 Whitening and redundancy reduction . . . . . . . . . . . . . . . . . . . . 126
5.9.2 Patch-based decorrelation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.9.3 Filter-based decorrelation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.10 Concluding remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
x Contents

5.11 Exercices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

6 Sparse coding and simple cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

6.1 Definition of sparseness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
6.2 Learning one feature by maximization of sparseness . . . . . . . . . . . . . 138
6.2.1 Measuring sparseness: General framework . . . . . . . . . . . . . . . 139
6.2.2 Measuring sparseness using kurtosis . . . . . . . . . . . . . . . . . . . . 140
6.2.3 Measuring sparseness using convex functions of square . . . . 141
6.2.4 The case of canonically preprocessed data . . . . . . . . . . . . . . . 144
6.2.5 One feature learned from natural images . . . . . . . . . . . . . . . . . 145
6.3 Learning many features by maximization of sparseness . . . . . . . . . . . 145
6.3.1 Deflationary decorrelation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.3.2 Symmetric decorrelation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.3.3 Sparseness of feature vs. sparseness of representation . . . . . . 148
6.4 Sparse coding features for natural images . . . . . . . . . . . . . . . . . . . . . . 150
6.4.1 Full set of features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.4.2 Analysis of tuning properties . . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.5 How is sparseness useful? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
6.5.1 Bayesian modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
6.5.2 Neural modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
6.5.3 Metabolic economy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
6.6 Concluding remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
6.7 Exercices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

7 Independent component analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

7.1 Limitations of the sparse coding approach . . . . . . . . . . . . . . . . . . . . . . 159
7.2 Definition of ICA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
7.2.1 Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
7.2.2 Generative model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
7.2.3 Model for preprocessed data . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
7.3 Insufficiency of second-order information . . . . . . . . . . . . . . . . . . . . . . 162
7.3.1 Why whitening does not find independent components . . . . . 163
7.3.2 Why components have to be non-gaussian . . . . . . . . . . . . . . . 164
7.4 The probability density defined by ICA . . . . . . . . . . . . . . . . . . . . . . . . 166
7.5 Maximum likelihood estimation in ICA . . . . . . . . . . . . . . . . . . . . . . . . 167
7.6 Results on natural images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7.6.1 Estimation of features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7.6.2 Image synthesis using ICA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
7.7 Connection to maximization of sparseness . . . . . . . . . . . . . . . . . . . . . . 170
7.7.1 Likelihood as a measure of sparseness . . . . . . . . . . . . . . . . . . . 171
7.7.2 Optimal sparseness measures . . . . . . . . . . . . . . . . . . . . . . . . . . 172
7.8 Why are independent components sparse? . . . . . . . . . . . . . . . . . . . . . . 175
7.8.1 Different forms of non-gaussianity . . . . . . . . . . . . . . . . . . . . . . 175
7.8.2 Non-gaussianity in natural images . . . . . . . . . . . . . . . . . . . . . . 176
7.8.3 Why is sparseness dominant? . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Contents xi

7.9 General ICA as maximization of non-gaussianity . . . . . . . . . . . . . . . . 177

7.9.1 Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
7.9.2 “Non-gaussian is independent” . . . . . . . . . . . . . . . . . . . . . . . . . 178
7.9.3 Sparse coding as a special case of ICA . . . . . . . . . . . . . . . . . . 179
7.10 Receptive fields vs. feature vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
7.11 Problem of inversion of preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . 181
7.12 Frequency channels and ICA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
7.13 Concluding remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
7.14 Exercices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

8 Information-theoretic interpretations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

8.1 Basic motivation for information theory . . . . . . . . . . . . . . . . . . . . . . . . 185
8.1.1 Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
8.1.2 Transmission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
8.2 Entropy as a measure of uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
8.2.1 Definition of entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
8.2.2 Entropy as minimum coding length . . . . . . . . . . . . . . . . . . . . . 189
8.2.3 Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
8.2.4 Differential entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
8.2.5 Maximum entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
8.3 Mutual information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
8.4 Minimum entropy coding of natural images . . . . . . . . . . . . . . . . . . . . . 194
8.4.1 Image compression and sparse coding . . . . . . . . . . . . . . . . . . . 194
8.4.2 Mutual information and sparse coding . . . . . . . . . . . . . . . . . . . 195
8.4.3 Minimum entropy coding in the cortex . . . . . . . . . . . . . . . . . . 196
8.5 Information transmission in the nervous system . . . . . . . . . . . . . . . . . 196
8.5.1 Definition of information flow and infomax . . . . . . . . . . . . . . 196
8.5.2 Basic infomax with linear neurons . . . . . . . . . . . . . . . . . . . . . . 197
8.5.3 Infomax with nonlinear neurons . . . . . . . . . . . . . . . . . . . . . . . . 198
8.5.4 Infomax with non-constant noise variance . . . . . . . . . . . . . . . 199
8.6 Caveats in application of information theory . . . . . . . . . . . . . . . . . . . . 202
8.7 Concluding remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
8.8 Exercices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

Part III Nonlinear features & dependency of linear features

9 Energy correlation of linear features & normalization . . . . . . . . . . . . . . 209

9.1 Why estimated independent components are not independent . . . . . . 209
9.1.1 Estimates vs. theoretical components . . . . . . . . . . . . . . . . . . . . 209
9.1.2 Counting the number of free parameters . . . . . . . . . . . . . . . . . 211
9.2 Correlations of squares of components in natural images . . . . . . . . . . 211
9.3 Modelling using a variance variable . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
9.4 Normalization of variance and contrast gain control . . . . . . . . . . . . . . 214
9.5 Physical and neurophysiological interpretations . . . . . . . . . . . . . . . . . 216
9.5.1 Cancelling the effect of changing lighting conditions . . . . . . 216
xii Contents

9.5.2 Uniform surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

9.5.3 Saturation of cell responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
9.6 Effect of normalization on ICA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
9.7 Concluding remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
9.8 Exercices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

10 Energy detectors and complex cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

10.1 Subspace model of invariant features . . . . . . . . . . . . . . . . . . . . . . . . . . 223
10.1.1 Why linear features are insufficient . . . . . . . . . . . . . . . . . . . . . 223
10.1.2 Subspaces or groups of linear features . . . . . . . . . . . . . . . . . . . 224
10.1.3 Energy model of feature detection . . . . . . . . . . . . . . . . . . . . . . 225
10.2 Maximizing sparseness in the energy model . . . . . . . . . . . . . . . . . . . . 227
10.2.1 Definition of sparseness of output . . . . . . . . . . . . . . . . . . . . . . 227
10.2.2 One feature learned from natural images . . . . . . . . . . . . . . . . . 228
10.3 Model of independent subspace analysis . . . . . . . . . . . . . . . . . . . . . . . 229
10.4 Dependency as energy correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
10.4.1 Why energy correlations are related to sparseness . . . . . . . . . 231
10.4.2 Spherical symmetry and changing variance . . . . . . . . . . . . . . . 231
10.4.3 Correlation of squares and convexity of nonlinearity . . . . . . . 232
10.5 Connection to contrast gain control . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
10.6 ISA as a nonlinear version of ICA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
10.7 Results on natural images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
10.7.1 Emergence of invariance to phase . . . . . . . . . . . . . . . . . . . . . . . 235
10.7.2 The importance of being invariant . . . . . . . . . . . . . . . . . . . . . . 242
10.7.3 Grouping of dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
10.7.4 Superiority of the model over ICA . . . . . . . . . . . . . . . . . . . . . . 243
10.8 Analysis of convexity and energy correlations* . . . . . . . . . . . . . . . . . . 245
10.8.1 Variance variable model gives convex h . . . . . . . . . . . . . . . . . . 246
10.8.2 Convex h typically implies positive energy correlations . . . . 246
10.9 Concluding remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
10.10Exercices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

11 Energy correlations and topographic organization . . . . . . . . . . . . . . . . . 249

11.1 Topography in the cortex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
11.2 Modelling topography by statistical dependence . . . . . . . . . . . . . . . . . 250
11.2.1 Topographic grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
11.2.2 Defining topography by statistical dependencies . . . . . . . . . . 251
11.3 Definition of topographic ICA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
11.4 Connection to independent subspaces and invariant features . . . . . . . 254
11.5 Utility of topography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
11.6 Estimation of topographic ICA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
11.7 Topographic ICA of natural images . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
11.7.1 Emergence of V1-like topography . . . . . . . . . . . . . . . . . . . . . . 257
11.7.2 Comparison with other models . . . . . . . . . . . . . . . . . . . . . . . . . 262
11.8 Learning both layers in a two-layer model * . . . . . . . . . . . . . . . . . . . . 264
Contents xiii

11.8.1 Generative vs. energy-based approach . . . . . . . . . . . . . . . . . . . 264

11.8.2 Definition of the generative model . . . . . . . . . . . . . . . . . . . . . . 264
11.8.3 Basic properties of the generative model . . . . . . . . . . . . . . . . . 265
11.8.4 Estimation of the generative model . . . . . . . . . . . . . . . . . . . . . 266
11.8.5 Energy-based two-layer models . . . . . . . . . . . . . . . . . . . . . . . . 270
11.9 Concluding remarks and References . . . . . . . . . . . . . . . . . . . . . . . . . . . 270

12 Dependencies of energy detectors: Beyond V1 . . . . . . . . . . . . . . . . . . . . . 273

12.1 Predictive modelling of extrastriate cortex . . . . . . . . . . . . . . . . . . . . . . 273
12.2 Simulation of V1 by a fixed two-layer model . . . . . . . . . . . . . . . . . . . 274
12.3 Learning the third layer by another ICA model . . . . . . . . . . . . . . . . . . 276
12.4 Methods for analysing higher-order components . . . . . . . . . . . . . . . . . 277
12.5 Results on natural images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
12.5.1 Emergence of collinear contour units . . . . . . . . . . . . . . . . . . . . 278
12.5.2 Emergence of pooling over frequencies . . . . . . . . . . . . . . . . . . 279
12.6 Discussion of results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
12.6.1 Why coding of contours? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
12.6.2 Frequency channels and edges . . . . . . . . . . . . . . . . . . . . . . . . . 285
12.6.3 Towards predictive modelling . . . . . . . . . . . . . . . . . . . . . . . . . . 285
12.6.4 References and related work . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
12.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

13 Overcomplete and non-negative models . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

13.1 Overcomplete bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
13.1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
13.1.2 Definition of generative model . . . . . . . . . . . . . . . . . . . . . . . . . 290
13.1.3 Nonlinear computation of the basis coefficients . . . . . . . . . . . 291
13.1.4 Estimation of the basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
13.1.5 Approach using energy-based models . . . . . . . . . . . . . . . . . . . 294
13.1.6 Results on natural images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
13.1.7 Markov Random Field models * . . . . . . . . . . . . . . . . . . . . . . . . 297
13.2 Non-negative models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
13.2.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
13.2.2 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
13.2.3 Adding sparseness constraints . . . . . . . . . . . . . . . . . . . . . . . . . . 302
13.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

14 Lateral interactions and feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

14.1 Feedback as Bayesian inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
14.1.1 Example: contour integrator units . . . . . . . . . . . . . . . . . . . . . . . 308
14.1.2 Thresholding (shrinkage) of a sparse code . . . . . . . . . . . . . . . 311
14.1.3 Categorization and top-down feedback . . . . . . . . . . . . . . . . . . 313
14.2 Overcomplete basis and end-stopping . . . . . . . . . . . . . . . . . . . . . . . . . . 315
14.3 Predictive coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
14.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
xiv Contents

Part IV Time, colour and stereo

15 Colour and stereo images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321

15.1 Colour image experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
15.1.1 Choice of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
15.1.2 Preprocessing and PCA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
15.1.3 ICA results and discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
15.2 Stereo image experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328
15.2.1 Choice of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328
15.2.2 Preprocessing and PCA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
15.2.3 ICA results and discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
15.3 Further references . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
15.3.1 Colour and stereo images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
15.3.2 Other modalities, including audition . . . . . . . . . . . . . . . . . . . . 336
15.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336

16 Temporal sequences of natural images . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337

16.1 Natural image sequences and spatiotemporal filtering . . . . . . . . . . . . 337
16.2 Temporal and spatiotemporal receptive fields . . . . . . . . . . . . . . . . . . . 338
16.3 Second-order statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
16.3.1 Average spatiotemporal power spectrum . . . . . . . . . . . . . . . . . 341
16.3.2 The temporally decorrelating filter . . . . . . . . . . . . . . . . . . . . . . 345
16.4 Sparse coding and ICA of natural image sequences . . . . . . . . . . . . . . 347
16.5 Temporal coherence in spatial features . . . . . . . . . . . . . . . . . . . . . . . . . 349
16.5.1 Temporal coherence and invariant representation . . . . . . . . . . 349
16.5.2 Quantifying temporal coherence . . . . . . . . . . . . . . . . . . . . . . . . 349
16.5.3 Interpretation as generative model * . . . . . . . . . . . . . . . . . . . . . 351
16.5.4 Experiments on natural image sequences . . . . . . . . . . . . . . . . 352
16.5.5 Why Gabor-like features maximize temporal coherence . . . . 355
16.5.6 Control experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
16.6 Spatiotemporal energy correlations in linear features . . . . . . . . . . . . . 359
16.6.1 Definition of the model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
16.6.2 Estimation of the model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
16.6.3 Experiments on natural images . . . . . . . . . . . . . . . . . . . . . . . . . 361
16.6.4 Intuitive explanation of results . . . . . . . . . . . . . . . . . . . . . . . . . 364
16.7 Unifying model of spatiotemporal dependencies . . . . . . . . . . . . . . . . . 365
16.8 Features with minimal average temporal change . . . . . . . . . . . . . . . . . 367
16.8.1 Slow feature analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
16.8.2 Quadratic slow feature analysis . . . . . . . . . . . . . . . . . . . . . . . . 371
16.8.3 Sparse slow feature analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 374
16.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375

Part V Conclusion
Contents xv

17 Conclusion and future prospects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

17.1 Short overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
17.2 Open, or frequently asked, questions . . . . . . . . . . . . . . . . . . . . . . . . . . 381
17.2.1 What is the real learning principle in the brain? . . . . . . . . . . . 382
17.2.2 Nature vs. nurture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
17.2.3 How to model whole images . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
17.2.4 Are there clear-cut cell types? . . . . . . . . . . . . . . . . . . . . . . . . . . 384
17.2.5 How far can we go? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
17.3 Other mathematical models of images . . . . . . . . . . . . . . . . . . . . . . . . . 386
17.3.1 Scaling laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
17.3.2 Wavelet theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
17.3.3 Physically inspired models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
17.4 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388

Part VI Appendix: Supplementary mathematical tools

18 Optimization theory and algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393

18.1 Levels of modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
18.2 Gradient method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
18.2.1 Definition and meaning of gradient . . . . . . . . . . . . . . . . . . . . . 395
18.2.2 Gradient and optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
18.2.3 Optimization of function of matrix . . . . . . . . . . . . . . . . . . . . . . 397
18.2.4 Constrained optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
18.3 Global and local maxima . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
18.4 Hebb’s rule and gradient methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
18.4.1 Hebb’s rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
18.4.2 Hebb’s rule and optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 401
18.4.3 Stochastic gradient methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 402
18.4.4 Role of the Hebbian nonlinearity . . . . . . . . . . . . . . . . . . . . . . . 403
18.4.5 Receptive fields vs. synaptic strengths . . . . . . . . . . . . . . . . . . . 404
18.4.6 The problem of feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
18.5 Optimization in topographic ICA * . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405
18.6 Beyond basic gradient methods * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
18.6.1 Newton’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
18.6.2 Conjugate gradient methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
18.7 FastICA, a fixed-point algorithm for ICA . . . . . . . . . . . . . . . . . . . . . . . 411
18.7.1 The FastICA algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
18.7.2 Choice of the FastICA nonlinearity . . . . . . . . . . . . . . . . . . . . . 411
18.7.3 Mathematics of FastICA * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412

19 Crash course on linear algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415

19.1 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
19.2 Linear transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416
19.3 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
19.4 Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
xvi Contents

19.5 Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419

19.6 Basis representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
19.7 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420
19.8 Pseudo-inverse * . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421

20 The discrete Fourier transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423

20.1 Linear shift-invariant systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
20.2 One-dimensional discrete Fourier transform . . . . . . . . . . . . . . . . . . . . 424
20.2.1 Euler’s formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424
20.2.2 Representation in complex exponentials . . . . . . . . . . . . . . . . . 425
20.2.3 The discrete Fourier transform and its inverse . . . . . . . . . . . . 428
20.3 Two- and three-dimensional discrete Fourier transforms . . . . . . . . . . 433

21 Estimation of non-normalized statistical models . . . . . . . . . . . . . . . . . . . 437

21.1 Non-normalized statistical models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
21.2 Estimation by score matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
21.3 Example 1: Multivariate gaussian density . . . . . . . . . . . . . . . . . . . . . . 440
21.4 Example 2: Estimation of basic ICA model . . . . . . . . . . . . . . . . . . . . . 442
21.5 Example 3: Estimation of an overcomplete ICA model . . . . . . . . . . . 443
21.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
Preface

Aims and scope

This book is both an introductory textbook and a research monograph on modelling

the statistical structure of natural images. In very simple terms, “natural images” are
photographs of the typical environment where we live. In this book, their statistical
structure is described using a number of statistical models whose parameters are
estimated from image samples.
Our main motivation for exploring natural image statistics is computational mod-
elling of biological visual systems. A theoretical framework which is gaining more
and more support considers the properties of the visual system to be reflections of
the statistical structure of natural images, because of evolutionary adaptation pro-
cesses. Another motivation for natural image statistics research is in computer sci-
ence and engineering, where it helps in development of better image processing and
computer vision methods.
While research on natural image statistics has been growing rapidly since the
mid-1990’s, no attempt has been made to cover the field in a single book, providing
a unified view of the different models and approaches. This book attempts to do just
that. Furthermore, our aim is to provide an accessible introduction to the field for
students in related disciplines.
However, not all aspects of such a large field of study can be completely covered
in a single book, so we have had to make some choices. Basically, we concentrate
on the neural modelling approaches at the expense of engineering applications. Fur-
thermore, those topics on which the authors themselves have been doing research
are, inevitably, given more emphasis.

xvii
Other documents randomly have
different content
The Project Gutenberg eBook of Collision Orbit
This ebook is for the use of anyone anywhere in the United States
and most other parts of the world at no cost and with almost no
restrictions whatsoever. You may copy it, give it away or re-use it
under the terms of the Project Gutenberg License included with this
ebook or online at www.gutenberg.org. If you are not located in the
United States, you will have to check the laws of the country where
you are located before using this eBook.

Title: Collision Orbit

Author: Clyde Beck

Illustrator: Herman B. Vestal

Release date: March 7, 2021 [eBook #64746]

Most recently updated: October 18, 2024

Language: English

Credits: Greg Weeks, Mary Meehan and the Online Distributed

Proofreading Team at http://www.pgdp.net

*** START OF THE PROJECT GUTENBERG EBOOK COLLISION

ORBIT ***
COLLISION ORBIT
 by CLYDE BECK

The tiny asteroid with the frightened girl

and the wrecked spacer with the grim young
man slowly spun closer and closer ... but
the real danger came after the crash!

[Transcriber's Note: This etext was produced from

Planet Stories Summer 1950.
Extensive research did not uncover any evidence that
the U.S. copyright on this publication was renewed.]
There's one good thing about a blowout. You don't need a mechanic
to tell you what the trouble is when it happens. This was the first
blowout I ever had, but as soon as I heard that explosive pinging
whistle and felt the floppy jolting and the terrifying sensation of a
vehicle out of control, I knew what was wrong. I reached forward
and cut the power.
When I leaned back in my seat I was sweating and my stomach was
pushing my tonsils around, and not only on account of the sudden
switch from one and a half G's to free fall. I was in a jam, and I
didn't need a mechanic to tell me that, either. Spaceships don't carry
spare drive tubes.
Not little wagons like the Aspera, anyway. If you could get a spare
inside the hull you would have to leave out the air plant or the
groceries or else stay home yourself, and even then there would be
no room for the tools to make the change. Retubing is a dock job,
and the nearest docks were a million miles away on Phobos and
getting farther fast.
And besides, you never need a spare. Tubes don't blow in space.
Diamondized graphite is tough—you caliper the throat every time
you dock, and after a few thousand G-hours you find enough erosion
to cut down efficiency to the point where it's a good idea to put in a
new liner.
I knew all this, but at the same time I knew the main tube had
blown. What I didn't know was what I was going to do about it. I lit
a cigarette and took a deep drag, just in case the stimulating effect
of the quabba smoke would give me an inspiration.
It made me sneeze.
I threw the butt on the deck and mashed it with my heel before it
could bounce off and go adrift in the cabin. I never had liked the
taste of the weedy stuff anyway. Smoking quabba is the prime
attribute of a spaceman—it has the reputation of being a specific
against spacesickness, toughening the cerebral meninges against
high acceleration, cutting down reaction time when you have to act
fast in a meteor field. Maybe it's all true. One thing it really does is
make your clothes smell like a vacant lot on fire so people can say,
"Ah, he's a spaceman," without having to ask.
No inspiration. Okay, Denby, think it out with your own brains.
You've got a brain, haven't you?
Not being very eager to do any thinking about the situation I was in,
I dragged the bulger out from under the seat and crawled into it. I
had a vague idea that I might fake up some sort of patch for the
tube and maybe limp back to Mars. I wasn't proud of it, but it was
the best I had at the moment. I checked to make sure there was
nothing on the screens, and then pulled myself over to the air lock,
sealed the inner door, and started the pump.
While the chamber was exhausting, I tested the lubber line and
snapped the end of it to a ring on the inner skin of the hull. When
the lock clicked I pulled the hatch open and hooked it back. Then I
took a short hold on the lubber line and stepped out into space.
For a minute I wished I had finished the quabba. This was not the
first time I had been in open space, but the circumstances had not
been so impressive before. Free fall had never bothered me
particularly, but it bothered me now, with millions of miles of empty
space under me in all directions and nothing in the sky but the tiny
hard bright stars looking very far away. And the realization that I
was alone, with a crippled ship, and a very good chance that the
situation would be permanent, made me feel that an antidote
against spacesickness would be a handy thing to have.
After a while the muscles of my forearm began to ache from
gripping the lubber line so hard. I let go of it and took hold of a
hand rail and crawled back to the stern.
It was a blowout, all right. The liner was completely gone and the
jacket was a fused lump of slag. All I would need to patch it up was
a week in the shops and a three-man crew. I crawled back along the
hull and went through the hatch like a rabbit going down its hole.
I stowed away the suit and belted myself in the seat. So I would
have to think anyway. I got out a pencil and reeled the tape out of
the accelerometer and began figuring.
It took me an hour, which was not very good. Neither was the
answer. I pushed the papers away and started all over again. The
answer was still the same. The Aspera would miss the orbit of
Jupiter by more than fifty million miles, and my nearest approach
would occur about three and a half years after Jupiter had passed
my intended point of tangency.
Of course these figures were only rough, and would be revised one
way or the other after I had time to make a few triangulation shots.
But I couldn't hope for much encouragement from any such revision.
The Aspera, the ship my father had used to make the first landing
on an asteroid ten years ago, was going to end up as an asteroid
herself, and I would have the honor of being sole inhabitant—as long
as I lasted.
I grabbed a sheet of paper and began figuring again. It took me
only a minute or two this time. The period of the Aspera's orbit was
seven and a half years, and seven and a half years Earth time make
four Mars years within a few days. That was how much hope I had—
in seven and a half years I would be back in the immediate vicinity
of Mars, and I might have enough power in the steering jets to claw
my way in to one of the moons. If I didn't bump into an asteroid. If
Jupiter didn't pull me too far off course. If I didn't go star-happy in
the meanwhile, or starve. Before seven and a half years were up I'd
be eating the air plant.
I threw down the pencil, caught it on the first wild bounce, and
stowed it away in my pocket. I felt like a fool.
With reason. It takes a very fancy kind of fool to rot four years in the
Girdle swamps on Venus, getting drunk only every second month so
he can save up enough of his pay to put himself through Space
Tech, and then, when he has graduated second in his class, to throw
away a plushy job with Translunar and go barging off into space in
an ancient can and get himself wrecked just because he lets a girl
talk him into making a magnificent gesture.
That's what I told myself. It didn't help any, but I had it coming. I
was a worse fool than that, even. Betty Day hadn't talked me into
this. I had thought the whole thing up with my own little brain. The
germ of the idea was hers, though, or rather the inspiration for it.

II

For that matter, Betty Day inspired a lot of my ideas, ever since my
first opening day at Space Tech. The first task they put us to on the
opening day was to sit through a welcoming address from the
President of the Institute. Maybe it was a good speech if you
happened to be a kid fresh out of school, like most of the class, with
your head full of the ideas of romance and glory that the tridim
space operas pump into the cash customers, but when he began to
talk about our "mission" and being "pioneers of the new frontier" it
got a little too thick for me.
I hadn't come to the Institute of Space Technology to look for glory.
I had come for the excellent if commonplace purpose of qualifying
for a well-paid job. My father's happy-go-lucky space-ratting was not
for me. I intended to do my planeteering with the resources of a
nice fat soulless corporation behind me. Four years in the Girdle of
Venus—which name, in case you are wondering, is a neat little piece
of irony—had left me very sane and practical and disenchanted
about the whole matter.
I let the President gabble on and began to glance around the
auditorium.
I didn't glance far. As I turned my face toward the girl sitting at my
left, she turned hers, and our eyes met. I managed a smile and
cocked an eyebrow toward the speaker's stand. She smiled back
with her eyes and crinkled her nose. It was a smooth straight nose,
and the eyes on each side of it were a clear cool gray, set well apart
under level brows. That was Betty—level and straight, and cool, too,
for that matter. I didn't realize all this at once, of course. Just now I
only knew that she was calmly and compellingly beautiful, and that I
didn't feel sane and practical any more, and certainly not
disenchanted.
There was a spatter of mildly enthusiastic applause, and I noticed
the lecture hall again and saw that the President had finished and a
youngish instructor was taking the stand to give out information
about programs and class assignments. I got down enough to keep
from getting lost. I heard him say the sections would be arranged
alphabetically. That scared me—suppose this girl was named
Wigglesworth or Zilch or some such and I would never see her
again! I drew a circle around my name on the class roster they had
given each of us at the beginning of the festivities and handed it to
her. She smiled again and drew a circle around the name right next
to it. Betty Day. So that was all right.

There is no time for social life at Space Tech. You go there for the
training and you get your money's worth. Not that I cared—the work
was hard, but it was exciting, and you could see the purpose of it as
you went along. I would have worked even harder and not minded,
because Betty Day was alongside in every class I had. After a few
days we were eating lunch together every day in the campus slop
shop, which arrangement I liked. It took my mind off the sort of
food they served there.
Every two or three weeks we found or took time to see a tridim
together, since there is not much else in the way of extracurricular
diversion at Tech. It was a very slight intimacy, but it meant a good
deal to me, and I believed that it did to Betty too. She was always
pleased to have me around, and she crinkled her nose at my jokes in
a special way that she did for no one else's, and my jokes were not
much better than the average, either.
It was a long time before I tried to tell her about the way I felt. It
was not until the three years at Tech were over and the Institute
was letting down its hair to the extent of sealing our brow with the
traditional farewell party for graduates known as the Blastoff.
By the time I got there the revelry had already started. I made a
couple of passes at the punch bowl and looked around for Betty. She
was out on the floor; I pried her loose from the Joe who was trying
to dance with her, and we made one eccentric ellipse around the hall
and headed for the terrace. It was cool out there, the unostentatious
coolness of an early summer evening that has not quite forgotten
the heat of the day, and there was a bright wash of moonlight on
the bay beyond the lights of the town. There was a lot of stardust
around.
Betty must have seen it too. She turned toward me, and the solemn
look on her face and the way her shoulders glowed in the moonlight
and the moonlight gleamed in her hair was enough to make your
breath come short. My breath, at least. It came right up in my throat
and stuck there, and I reached out and we sort of melted together.
It was the first time that had happened. That's how hard they work
you at Tech.
After a little while we separated and I opened my eyes and they still
worked well enough for me to see a bench not far away and we
walked over and sat down.
Betty sighed and leaned toward me and I moved my arm out of the
way to make room. The skin of her shoulder was smooth to my
hand, and cool the way the evening air was cool.
"It's been fun, Tom, hasn't it?" I knew she meant the last three
years and not just the last three minutes.
"Lots of work and lots of fun," I agreed. "That's why space work
gets in your blood, I think. It's fun even when it's hardest. My hitch
in the Girdle even seems like fun now that it's over."
"I can see how planet work must be a thrill, even if I haven't ever
been beyond the moon. I will be though—I'm going out with my
uncle's Vesta expedition in a couple of weeks, you know."

I hadn't known. I knew she had been talking about it, but I had
hoped Ed Day would have sense enough to say no. I wasn't
altogether selfish about it. I did want her closer in, nearer where I
would be, but a big part of the reason was that the asteroid belt was
the Edge, and the Edge has always been a rough place for women,
even when it was at the moon.
I started to tell her this, but she interrupted. "How did you make out
with Translunar? The man must have had a lot to say to keep you
this long."
"I get the money all right. And a job."
"A good one?"
"Six thousand."
"Yes, but what and where?"
"Luna City. I'll be port engineer."
"Oh, Tom!" I didn't think she had to put so much disappointment in
her voice. It was practically disdain. "I should think Translunar could
do better than that. It's practically landlocked. You aren't going to
take it, are you?"
"Why not? Six thousand is a nice sackful of cash, and besides, I get
a piece of the company. Not a very big one, but it will grow."
"Oh, Tom!" It was pure disdain this time. "It isn't the money! You
should have a ship. You should be out doing things. They can't make
you into a glorified slug monkey on the moon!" She pulled away
from my arm and looked at me again. The solemn expression on her
face was somber now, or maybe sullen; anyway I didn't like it.
"For six thousand they can do worse than that," I said. "It's more
than the captain of a liner gets. And, anyway, Translunar's ships are
all staffed. There wouldn't be a place for me even if I wanted one,
and I'm not sure I want one. Maybe there's more glamour in being a
deep-space man, but you can't call the job the engineers do trivial.
The idea of being a slug monkey doesn't bother me at all at that
pay. It's better than being a swamp hog on Venus."
"But it's such a waste, Tom! Anyone can be an engineer. You should
be in research or exploration, and you know it. It's a crime to waste
your talent in a dock job. You belong out on the Edge."
"Look, Betty—there are three sorts of Edge jobs: in the Patrol, on
some sort of an expedition, or as a space-rat. The first two don't pay
and, as for the third, even if I liked the idea of prospecting the
planets, it takes money to outfit for it, and it took all I had to finish
Tech."
"But you have the Aspera, and the Translunar prize would be enough
to get her into shape again and buy supplies."
"I was given to understand this afternoon that it would be
considered very unconventional to take the money and not take the
job. And anyway, what would I do then—hunt for thorium in the
asteroids? No thanks. I'll take the slug monkey job and the salary.
And I think you ought to do the same. You could get a job closer in
that would pay a lot more than going off to the Belt on a wild goose
chase. When you graduate first in your class at Tech you can take
your pick."
"Wild goose chase!" She sniffed. "We are going out to get data on
the Warp at close range. We might even find out the way to get
around it and open up the outer planets to exploration."
The Warp was supposed to be a sort of fourth-dimensional wrinkle in
space somewhere beyond the asteroids that swallowed ships and
accounted for the fact that out of three expeditions that had tried to
reach Jupiter, three had not returned. I knew better.
"There isn't any Warp," I told her. "My father proved that eight years
ago when he made the swing around Jupiter."
"But he never published any proof, Tom."
"No, all the proof he had was in his log book, and that went with
him on his last trip. But I read the log. He sighted the pirate camp
on Callisto, and would have had pictures to prove it if all his film
hadn't been raystruck. Maybe he could have got somebody to listen
to him anyway if he had tried a little harder, but he wanted to make
a research job of it. He sold out all his claims and built the Astra and
loaded it up with equipment to bring back all the proof that even the
Patrol could ask for. Then he blasted off and no one ever heard of
him again."
"But the idea of pirates doesn't make sense, Tom. There are no
cargoes worth stealing beyond the Belt. On the Venus run, yes—but
why should there be pirates out where there are no ships?"
"Okay, no pirates, then. What they really are is Hassley and all those
hangers-on of his that were never accounted for after the Polar War.
One of the moons of Jupiter would make a fine hideout for them. Air,
water, and a livable climate. When any one comes snooping around,
they see to it that they never get back. We blame it on the Warp and
stay away and leave them alone."
"They would never get there in the first place. The Warp isn't just
somebody's wild guess, you know. It follows from Heuvelstad's work.
He derived Bode's law from quantum theory, and showed that a
warp in space is the only explanation for the family of asteroids
between Mars and Jupiter where there should be a single planet. No
one can doubt it."
"I can. No one used to doubt that the earth was flat, or to bring it a
little more up to date, that the craters of the moon were volcanoes,
or that the red shift in the nebular spectra meant that the universe is
expanding. A theory is good only as long as it explains all the facts,
and Heuvelstad overlooked the fact that my father circled Jupiter
and came back. He will just have to revise his mathematics."
"Maybe we'll know more about that after the Vesta expedition comes
back." She sighed and looked out over the glittering bay.
I sighed too, and took my arm away from the back of the seat. I
didn't quite know how the conversation had wandered so far from
the point. I had felt quite set up about everything when I came to
the party. I thought Betty would be glad about the Translunar offer,
and maybe remark that six thousand credits was a remarkable salary
for a fresh graduate, and I would suggest that it was enough to get
married on. And here we were arguing.
She turned and looked at me again. "Tom," she said softly; maybe I
was going to have my chance after all.
"Yes?" I answered.
"Are you really going to take that engineer job? Couldn't you talk
Translunar out of something that would give you the chance to do
the things a Denby ought to be doing?"
"Maybe I could. But look—I've sweated out the last seven years just
for the chance I've got right now, and I mean to take it. My father
spent all his life chasing a dream, and what did it get him? The one
great discovery he did make no one will even believe."
"I never met Lance Denby, but I know he was a great spaceman,
Tom, even if you do seem to have forgotten it. I never thought a son
of his would ever turn out to be a company man. Let's go inside."
We went inside, and I went home. The punch bowl was empty by
now so I didn't even stop.
It was probably a mistake, but I flew down to Mojave Outport the
day the Vesta Expedition blasted off. Betty was very friendly when
we said goodbye, and her hand in mine was small and firm, and the
fingers were quite cold. I don't remember what I said. It couldn't
have been much. There was a stiff feeling around my lips that it was
hard to push any words through.
Betty was last on board. She turned and looked back for a few
seconds before they closed the hatch, and it seemed to me that
there was the same solemn expression on her face that I had seen
that night on the terrace. I was too far away to be sure.
My interview with the Western manager of Translunar was scheduled
for the next day. I'm afraid I made a poor impression from the very
start. I wasn't feeling very sharp; instead of sleeping I had spent a
good part of the night wondering about that look in Betty's eyes.
That and a few other things.
Elkins, the manager, was the sort of man who wears a nice sharp
crease in his pants and his hair brushed carefully over his bald spot
and calls everyone heartily by his first name.
"Well, Tom," he said expansively, after the formalities of introduction
and exchange of cigarettes were out of the way, "let's get to
business. First of all, this, ah—token."
He held out a check. The four figures on it were even prettier than
the pretty-colored ink they were printed in. That was for me. Legally,
by the terms of their prize offer. I had checked on that.
"Thank you," I said.
"And now, as concerns your place with the Translunar organization
—"
I interrupted. "I'm sorry, Mr. Elkins. Personal plans make it
impossible for me to accept the position you have so generously
offered me."
That rocked him. Why not—it rocked me. He still smiled with his lips,
out of habit, but his eyes weren't smiling. He pulled an ash tray to
him and crushed out his cigarette—the one I had given him.
"But—! You realize this is most irregular, Mr. Denby! And
unexpected."
"I do. I didn't know it myself until a little while ago."
"Is this decision final, Mr. Denby?"
"I'm afraid it is."
"Very well. I'm sorry to hear it." His tone meant that I would be
sorry too. "In that case there is nothing further to say."
He pushed a button and a flunky came in to sweep me out. As I left
I could as good as see him writing down my name on a sheet
marked Blacklist in 72 point caps.

III

It took three months to make the Aspera spaceworthy again, and

when I had bought the shielded tele-camera, vitanalyzer, and the
other little toys I would need to prove that there was a pirate
hideout in Callisto, my bank account was within saluting distance of
absolute zero. This was space-ratting for fair, without even a chance
for paydirt at the end of the orbit. And Translunar, or any other
outfit, wouldn't have me even as a swamp hog after this. I was the
smart Joe who was going to have me a Career.
You never know.
I stopped at Phobos to fill up on reactant. I didn't mean to land on
Callisto. I didn't even mean to be seen if I could help it, but still I
might have some dodging to do, and full tanks could be nice to
have. For the same reason I put in a new power slug, because the
emission had begun to go a little soft on the way out from Luna City.
With the salvage of the old one, that left me just enough for a
couple of highballs at the port canteen. I thought I needed them
more than two loose coins. I left the slug monkey grumbling about
having to root around among the obsolete parts to find a Group VI
slug, and headed for the bar. Let him grumble. The Aspera was still a
good ship, even if she didn't have the tungsil tubes it takes to handle
Group IV fissionables.
Wait a minute! Maybe he got tired of rooting and put in a Group IV
slug just out of laziness and ignorance. I made my way back to the
power shack, cracked the case, and took a look through the
periscope. The IV on the can was as big as a house. Well, when I
got back I would be able to prove to Betty that I was right about
trained personnel not being wasted in the engineering department.
If I got back.
Seven and a half years in a space can is a horrible thought, but to
do it in free fall is out of the question. I swung a pair of steering
rockets to tangent position and cranked up enough rotation to give
me a few pounds of weight. That made a mess out of the visual
screens, but the radek would still let me know if anything came close
enough to worry about, and this way a cup of coffee would at least
stay in the cup. I brewed a pot of it, stuffed a pipe full of tobacco,
and started to settle down to do my time.
I don't know how many days later it was that the radek began to
groan. I quit counting days after the first week—if I needed the date
I could get it off the chronograph. The signal was feeble, but I took
the twist off her to get a fix on what it was. The radek gave the
range as extreme—nearly a million miles—and anything that would
trip the relay at that range must be big. After a few sweeps I found
it in the scope, and it showed a perceptible disk. That meant an
asteroid. I didn't know which one—the General Emphemeris of the
asteroids hasn't been published yet.

During the next day or two I spent a good deal of my time at the
scope, and most of the rest figuring orbits. It was pleasant to have
something to do to keep my mind off my predicament. I hardly
minded even when it became obvious that I would come so close to
the asteroid as to be perturbed out of all possibility of making the
contact with Mars that I had projected. I hadn't really believed in
that anyway. And, when I discovered that I was in a collision orbit, it
was more of a relief than otherwise. Get it over with in a hurry.
Starvation is a slow and tedious way to blast off. A short life and a
merry one, Denby, that's what you always said. Or did you? Well, it
doesn't matter, you're going to get it anyway.
It was a fine sight. I don't know anything more impressive to watch
than a planet, even a little two-hundred-mile chunk of rock like this
one, swinging up out of empty space and taking on size and form.
White and round as a snowball, and spinning lazily like a snowball
thrown through the air. This one was going to hit me right on the
knob.
The twelve-hour rotation of the asteroid must have swung the spot
past me three or four times before I paid any attention to it. A black
smudge it was, round, but with ragged edges like a starfish. A jet
scorch if I ever saw one. I swallowed my stomach on the third gulp,
and as soon as I stopped being dizzy I looked again. A jet scorch it
was, and a few hundred yards away the sunlight glittered on a round
lump that couldn't be anything but a Mitchell blister. Of all the rocks
in the Belt, I would bump into one with a station on it. Nice catch,
Denby!
I crawled into the bulger again in case I might set her down a little
heavy, and got at the controls. Landing on the steering jets is tricky,
especially when there is no atmosphere to help you brake down. I
never would have made it if it had been a full-sized planet.
I set her down heavy, all right, but I'm not ashamed of it. Try it
yourself some time. We crashed in a gully some sixty feet deep,
about a mile from the station. The shock broke my belt and threw
me against the control panel, and I felt a couple of ribs crack. That
was cheap. When my head cleared a little I could hear rocks rattling
on the hull and air whistling out through a hole in her somewhere. I
made a dash for the lock and kicked the emergency hatch release
and blew outside with the rest of the air.

Just in time. Looking up, I could see the whole side of the cliff
coming loose and toppling toward me like the crest of a breaker. I
gritted my teeth and jumped. When I looked back there was nothing
to see but a heap of rock.
Under this light gravity, the leap took me well above the cliffs. I
could see a glint of sunlight on the Mitchell in the distance, and a
spacesuit-clad figure coming over the surface in long leaps. One
jump had been enough for me—I hung onto my ribs and did my best
to walk. That isn't easy with a gravity a couple of hundredths Earth
normal, but at least when you fall you don't hit very hard.
In a minute or two I came up with my rescuer, and we touched
helmets to talk. I stared through the faceplate of the other suit.
"Hello, Betty," I said. Then I passed out.

When I woke up someone was swabbing my face with a damp cloth.

It was very pleasant. I opened my eyes, and it was Betty, all right.
"Hello yourself," she said, and smiled. It was the old smile, crinkled
nose and all. I took back what I had told myself about being a fool. I
sat up and reached out my arms, but the ribs got in the way.
"Tom!" she cried. "What's the matter?"
"I bent a couple of ribs a little too far," I answered. "Nothing vital."
"Here, let me help!"
Between us we pulled the bulger off me and got rid of my packet
and shirt. Betty crossed the room and began to rummage in a locker.
I looked around. I was on a folding cot in one of the sleeping
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookball.com