2921 anaes ‘Automatic Image Quality Assessment in Python | by Ricardo Ocampo | Towards Data Science Open in app 7 Medium @ search a This member-only story is on us. Upgrade to access all of Medium, + Member-only story Automatic Image Quality Assessment in Python Ricardo Ocampo - Follow @® Puviished in Towards Data Science Tmin read - Aug 29, 2018 © Listen (1) Share + More Image quality is a notion that highly depends on observers. Generally, it is linked to the conditions in which it is viewed; therefore, it is a highly subjective topic. Image quality assessment aims to quantitatively represent the human perception of quality. These metrics are commonly used to analyze the performance of algorithms in different fields of computer vision like image compression, image transmission, and image processing [1]. Image quality assessment (IQA) is mainly divided into two areas of research (1) reference-based evaluation and (2) no-reference evaluation. The main difference is that reference-based methods depend on a high-quality image as a source to evaluate the difference between images. An example of reference-based evaluations is the Structural Similarity Index (SSIM) [2]. No-reference Image Quality Assessment No-reference image quality assessment does not require a base image to evaluate image quality, the only information that the algorithm receives is a distorted image whose quality is being assessed. Blind methods are mostly comprised of two steps. The first step calculates features that describe the image's structure and the second step finds the patterns among the hitps:Rowardsdatascionce.comlautomatic-image-qualty-assossment-inpython-391a6beS2c11 1s.aac ears ‘Auta Inage Qual Assessment in Python by Reardo Ocampo | Towards Data Sience features to human opinion. TID2008 is a famous database created following a methodology that describes how to measure human opinion scores from referenced images [3]. It is widely used to compare the performance of IQA algorithms. For an implementation of a deep learning method using TensorFlow 2.0 check: Deep Image Quality Assessment with TensorFlow 2.0 Astep by step guide to implement the Deep CNN-Based Blind Image Quality Predictor (DIQA) with TensorFlow 2.0 medium.com Blind/referenceless image spatial quality evaluator (BRISQUE) In this section, we will code step by step how the BRISQUE method in python. You can find the complete notebook here. BRISQUE [4] is a model that only uses the image pixels to calculate features (other methods are based on image transformation to other spaces like wavelet or DCT). It is demonstrated to be highly efficient as it does not need any transformation to calculate its features. BRISQUE relies on spatial Natural Scene Statistics (NSS) model of locally normalized luminance coefficients in the spatial domain, as well as the model for pairwise products of these coefficients. Natural Scene Statistics in the Spatial Domain Given an image, we need to compute the locally normalized luminescence via local mean subtraction and divide it by the local deviation. A constant is added to avoid zero divisions. wy _ Lind) — mli.3) N= TE +e *Hint: If (i,j) domain is [0, 255] then C=1 if the domain is (0, 1] then C=1/255. To calculate the locally normalized luminescence, also known as mean subtracted contrast normalized (MSCN) coefficients, we have to calculate the local mean. Here, wis a Gaussian kernel of size (K, L). ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 262921 anaes ‘Automatic Image Quality Assessment in Python | by Ricardo Ocampo | Towards Data Science K OL The way that the author displays the local mean could be a little bit confusing but it is calculated by just applying a Gaussian filter to the image. def normalize_kernel(kernel): return kernel / np.sum(kernel) def gaussian_kernel2d(n, sigma): Y, X = np.indices((n, n)) = int(n/2) gaussian_kernel = 1 / (2 * np.pi * signa ** 2) * np.exp(-(x ** 2+ Y ** 2) / (2 * signa ** 2)) return normalize_kernel (gaussian_kernel) def local_mean(image, kernel): return signal.convolve2d(inage, kernel, ‘sane’) Then, we calculate the local deviation KOOL SS > wea Tea(t, J) — HF) Kl=-L def local_deviation(image, local_mean, kernel): “Vectorized approximation of local deviation” signa = image ** 2 signa = signal.convolve2d(signa, kernel, ‘sane') return np.sqrt(np.abs(local_mean ** 2 - sigma)) Finally, we calculate the MSCN coefficients def calculate_mscn_coefficients(image, kernel_size=6, signa=7/6) C= 1/255 kernel = gaussian _kernel2d(kernel_size, sigma=signa) local_mean = signal.convolvezd(image, kernel, ‘sane’) local_var = local_deviation(image, local_mean, kernel) return (image - local_nean) / (local_var + C) ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 382921 anaes ‘Automatic Image Quality Assessment in Python | by Ricardo Ocampo | Towards Data Science The author found that the MSCN coefficients are distributed as a Generalized Gaussian Distribution (GGD) for a broader spectrum of the distorted image. The GGD density function is f(a;a,o) where and I is the gamma function. The parameter a controls the shape and o” the variance. def generalized_gaussian_dist(x, alpha, signa) beta = signa * np.sqrt(special.ganma(1 / alpha) / special.ganma(3 / alpha) coefficient = alpha / (2 * beta() * special.ganma(1 / alpha)) return coefficient * np.exp(-(np.abs(x) / beta) ** alpha) Pairwise products of neighboring MSCN coefficients The signs of adjacent coefficients also exhibit a regular structure, which gets disturbed in the presence of distortion. The author proposes the model of pairwise products of neighboring MSCN coefficients along four directions (1) horizontal H, (2) vertical V, (3) main-diagonal D1 and (4) secondary-diagonal D2. H(i,j) = 10,515 + V(i,j) = 1,5) LE + 1,3) DA(i,j) = Ti, j)Mi+ LF +1) D2i, 3) = (i,j) Fi + 1,5 — 1) ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 ans.2921 anaes ‘Automatic Image Quality Assessment in Python | by Ricardo Ocampo | Towards Data Science def calculate_pair_product_coefficients(mscn_coefficients) return collections orderedDict({ “mscn*: mscn_coefficients, ‘horizontal’: mscn_coefficients[:, ‘vertical’: mscn_coefficients[:-1, mscn_coefficients[1:, :1, *main_diagonal': mscn_coefficients[:-1, :-1] * mscn_coefficients[1:, 1:] “secondary_diagonal': mscn_coefficients[1:, :-1] * mscn_coefficients[ 1] * msen_coefficients| » Also, he mentions that the GGD does not provide a good fit to the empirical histograms of coefficient products. Thus, instead of fitting these coefficients to GGD, they propose to fit an Asymmetric Generalized Gaussian Distribution (AGGD) model [5]. The AGGD density function is —» __(-()") p20 Y,01,0r) = ov (ey) r>=0 + ay x f(x ¢ ay where and side can be either r or I. Another parameter that is not reflected in the previous formula is the mean r(j r( ) ) n = (8, — beta) def asymnetric_generalized_gaussian(x, nu, signal, signa_r) def beta(sigma): return signa * np.sqrt(special.ganna(1 / nu) / special.ganma(3 / nu)) coefficient = nu / ((beta(signa_l) + beta(signa_r)) * special.ganna(1 / nu)) # = lambda x, signa: coefficient * np.exp(-(x / beta(signa)) ** nu) return np.where(x < @, f(-x. signal), f(x, sigma_r)) Fitting Asymmetric Generalized Gaussian Distribution ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 518aac ears Automat Inage Qvaly Assessment nPthn by Ricardo Ocampo | Towards Data Sion The methodology to fit an Asymmetric Generalized Gaussian Distribution is described in [5]. In summary, the algorithm steps are: 1. Calculate y where N; is the number of negative samples and N, is the number of positive samples. 2. Calculate r hat. NtN, 3. Calculate R hat using y and r hat estimations. 4, Estimate a using the approximation of the inverse generalized Gaussian ratio. p(R) _ _ T(2/a)? 5, Estimate left and right scale parameters. — 1 Ne 22 1= VW N-T Veet ap<0 Te Nr 2 T Dict ep>=0 i, pl Oo, = ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 ens2921 anaes ‘Automatic Image Quality Assessment in Python | by Ricardo Ocampo | Towards Data Science dof asyrmatric_generalized goussian f3t (x) def catinate_phi(olphe) rnmerator = special. gowma(2 / alpha) * 2 denoninator = special gamma(1 / slpha) * speciel-gamea(3 / alpha) return numerator / denominator det estinate e natd Size = mp.prod(x.shape) return (np. sunlng.abs(x)) / size) = 2 / (rp.sun(x ** 2) / size) det estinate } Peestoatee hat(r_hat, gama) (gama ** 5+ 1) * (gama + 2) (games #02 9.1) 2 return r-hat * numerator / denowinator ef nzan_squares_sum(x, filter = tanbda 2 filtered values = 2[Filter(3)] squares_sum = np-sun(Filtered values * 2) return squares_sun / (Filtered values. shape)) ) wan _squares_sun( lambda zi 2 < 6) Imean_squares_sun(%, Lanbds 2: x >= 0) 2 deft squares Pight_squares| return op.sqrt(Ieft_squares) / np.sqrt(night_squares) hat (x) arma) vhst(r-bat, gamma) solution = optinize.roat(Iamds 2: extinste phi(z) = hat, (6.21) ekors wedersere def estinste sigea(s, alphe, filter = lambda z: 2 ¢ 0): return np-sart(mcan squares sun(x, filter)) det estinate mean(alpha, signe 1, signa r): return (signar = signal) * constant * (speciel-gamea(2 / alpha) / special.ganna(a / slphs)) Wpha s Gtieete apts} Siaw_1 = estinate signa(x, alpha, lambda z: 2 < 0) Sigua = estimate signa(, alpha, lambda 2: 2 >= 0) 2 / alpha) / spacial. ganma(2 / alpha)) feean = ectinate-noan(aiphay signa 1, signa.) return alpha, moon, signal, signa Calculate BRISQUE features The features needed to calculate the image quality are the result of fitting the MSCN coefficients and shifted products to the Generalized Gaussian Distributions. First, we need to fit the MSCN coefficients to the GDD, then the pairwise products to the AGGD. A summary of the features is the following: ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 78azarae ‘Automate mage Quay Assessment in Python |by Rleardo Ocampo | Towards Data Science Feature ID Feature Description Computation Procedure fi-h Shape and variance —_—Fit GGD to MSCN coefficients —fs Sr — fio fir — fia Sis — fis Shape, mean, left variance, right variance Fit AGGD to H pairwise products Shape, mean, left variance, right variance Fit AGGD to V pairwise products Shape, mean, left variance, right variance Fit AGGD to D1 pairwise products Shape, mean, left variance, right variance Fit AGGD to D2 pairwise products def calculate_brisque_features(image, kernel_size=7, signa=7/6) def calculate_features(coefficients_nane, coefficients, accum=np.array([])) alpha, mean, signal, signa_r = asymnetric_generalized gaussian_fit(coefficients) Af coefficients_nane =: var signal *t 2+ sigmar ** 2) / 2 return [alpha, var] return (alpha, mean, signal ** 2, sigma_r ** 2] mscn_coefficients = calculate_nscn_coefficients(image, kernel_size, signa) coefficients = calculate_pair_product_coefficients(nscn_coefficients) features = [calculate_features(nane, coeff) for name, coeff in coefficients. items()] Flatten_features = list (chain. fron_iterable(features)) return np.array(flatten_features) Hands-on After creating all the functions needed to calculate the BRISQUE features, we can estimate the image quality for a given image. In [4], they use an image that comes from the Kodak dataset [6], so we will use it here too. Auxiliary Functions def load_image(url): image_stream = request .urlopen(url) return skimage. io. imread(image_stream, plugin="pil’) def plot_histogram(x, label): nn, bins = np.histogram(x.ravel(), bins=50) n =n / np.max(n) plt.plot(bins[:-1], n, label=label, marker='0') 1. Load image ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 ane2921 anaes ‘Automatic Image Quality Assessment in Python | by Ricardo Ocampo | Towards Data Science Xaatplotlib inline plt.reParans[ url = ‘http://snm.cs-albany.edu/~xypan/res: ng/Kodak/kodim@S.. png’ image = load_image(url) gray_image = skimage.color.rgb2gray(image) skimage. io. imshow( image) 2. Calculate Coefficients xxtime nscn_coefficients = calculate _mscn_coefficients(gray_image, 7, 7/6) coefficients = calculate_pair_product_coefficients(mscn_coefficients) CPU times: user 90 ms, sys: @ ns, total: 96 ms Wall tine: 91.4 ms Xmatplotlib inline plt.reParams["figure.figsize"] = 12, 11 for name, coeff in coefficients. itens() plot_histogran(coeff.ravel(), name) plt-axis([-2.5, 2.5, @, 1.05]) pit. legend() pit. show() hitps:Rowardsdatascionce.comlautomatic-image-qualty-assossment-inpython-391a6beS2c11 ons2921 anaes ‘Automatic Image Quality Assessment in Python | by Ricardo Ocampo | Towards Data Science After calculating the MSCN coefficients and the pairwise products, we can verify that the distributions are in fact different. 10 teted 2 4 0 i 3. Fit Coefficients to Generalized Gaussian Distributions xxtine beisque_features = calculate_brisque_features(gray_image, kernel_size=7, signa=7/6) CPU times: user 140 ms, sys: @ ns, total: 140 ms Wall time: 138 ms 4. Resize image and Calculate BRISQUE Features ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 horizontal vertical ‘main_diagonal secondary. diagonal 062921 anaes ‘Automatic Image Quality Assessment in Python | by Ricardo Ocampo | Towards Data Science sxtine donnscaled_image = cv2.resize(gray_image, None, fx=1/2, fy=1/2, interpolation = cv2.INTER_CUBIC) dounscale_brisque_features = calculate_brisque features(downscaled_image, kernel_size=7, signa=7/6) brisque features = np.concatenate((brisque features, downscale_brisque_features)) CPU times: user 40 ms, sys: @ ns, total: 40 ms Wall tine: 36 ms 5. Scale Features and Feed the SVR The author provides a pre-trained SVR model to calculate the quality assessment. However, in order to have good results, we need to scale the features to [-1, 1]. For the latter, we need the same parameters as the author used to scale the features vector. def scale_features(features): with open(*normalize.pickle’, ‘rb') as handle: normalize params = pickle.1oad(handle) min_ ip-array(normalize_parans[‘nin_'] np.array(normalize_parans['nax_'] ) ) return -1 + (2.0 / (max_ - min_) * (features - min_)) def calculate_image_quality_score(brisque_features) model = svmutil.svm_load_nodel("brisque_svm.txt') scaled_brisque_features = scale_features(brisque_features) x, idx = svmutil.gen_svm_nodearray( scaled_brisque_features, iskernel=(nodel.paran.kernel_type svmut 11 ,PRECOMPUTED) ) nr_classifier = 1 probestinates = (svmutil.c_double * nr_classifier)() return svmutil.1ibsvm.svm_predict_probability(model, x, prob_estimates) The scale used to represent image quality goes from 0 to 100. An image quality of 100 means that the image’s quality is very bad. In the case of the analyzed image, we get that it is a good quality image. xXtine calculate_image_quality_score(brisque_features) CPU times: user 8 ns, sys: Wall time: 12.7 ms 1@ ms, total: 1¢ ms 4,954157281562374 Conclusion ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 a6aac ears ‘Auta Inage Qual Assessment in Python by Reardo Ocampo | Towards Data Sience This method was tested with the TID2008 database and performs well; even compared with referenced IQA methods. I would like to check the performance of other machine learning algorithms like XGBoost, LightGBM, for the pattern recognition step. Python Notebook Update 2019-12-02: Included support for Windows 10 ocampor/notebooks You can't perform that action at this time. You signed in with another tab or window. You signed out in another tab or. bitly References {1] Maitre, H. (2017). From Photon to pixel: the digital camera handbook. John Wiley & Sons. [2] Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4), 600-612. [3] Ponomarenko, N., Lukin, V., Zelensky, A., Egiazarian, K., Carli, M., & Battisti, F. (2009). T1D2008-a database for evaluation of full-reference visual quality assessment metrics. Advances of Modern Radioelectronics, 10(4), 30-45. [4] Mittal, A., Moorthy, A. K., & Bovik, A. C. (2012). No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing, 21(12), 4695-4708. (5] Lasmar, N. E., Stitou, ¥., & Berthoumieu, Y. (2009). Multiscale skewed heavy- tailed model for texture analysis. Proceedings — International Conference on Image Processing, ICIP, (1), 2281-2284. Machine Learning Image Quality Asessment Python Al Image Processing ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 262921 anaes ‘Automatic Image Quality Assessment in Python | by Ricardo Ocampo | Towards Data Science Cc) Published in Towards Data Science 773K Followers - Last published just now Your home for data science and Al. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Cc) Written by Ricardo Ocampo 234 Followers - 17 Following Data Scientist, | research and blog about machine learning in my spare time Responses (7) 9 ‘Shubhadeep Roychowdhury . over 6 years ago Thanks for this article. | am working on a project where | will be hugely benefited by using some of the techniques stated here. S22 Reply hitps:Rowardsdatascionce.comlautomatic-image-qualty-assossment-inpython-391a6beS2c11 13182921 anaes ‘Automatic Image Quality Assessment in Python | by Ricardo Ocampo | Towards Data Science Javeria Ejaz about 5 years ago Thank you for the article. It was a great help. Though | couldn't comprehend the reason of concatenating the brisque feature of resized image. Hope you'll have a time to explain it. &1 Reply Michocam over 1 year ago Thanks for the article Ss Reply C See all responses ) Recommended from Medium Yet |e SAT ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 saneaot stot nage Cully Asatonat in Pyton | lero ampo | Towed Dta Secs BD in Towards Data Science by Florent Poux, PhO The Blender Handbook for 3D Point Cloud Visualization and Rendering Complete guide to create 3D experiences with large point clouds in Blender + Feb28 W254 @ Wo {Bn DataDriveninvestor by Austin Starks lused OpenAl’s 01 model to develop a trading strategy. It is DESTROYING the market It literally took one try. | was shocked. + Sepi6 Wek @ 183 Predictive Modeling w/ Python 20stories - 1705s Practical Guides to Machine Learning 1Ostories . 2076 saves Coding & Development hitps:Rowardsdatascionce.comlautomatic-image-qualty-assossment-inpython-391a6beS2c11 156ani eras ‘Automate mage Quay Assessment in Python |by Rleardo Ocampo | Towards Data Science Yl Natural Language Processing | es Boson Psi Solving the Equations: Unveiling Challenges and Solutions for PDEs in Simulations Challenges and Solutions for PDEs in Simulations Juni2 2 hitps:Rowardsdatascionce.comlautomatic-image-qualty-assossment-inpython-391a6beS2c11 662921 anaes ‘Automatic Image Quality Assessment in Python | by Ricardo Ocampo | Towards Data Science Harendra How! Am Using a Lifetime 100% Free Server Get a server with 24 GB RAM + 4 CPU + 200 GB Storage + Always Free + Oct26 WESK @ 102 [iow In Stackademic by Abdur Rahman Python is No More The King of Data Science 5 Reasons Why Python is Losing Its Crown + Oct23 Wo2k @35 ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 a82921 anaes ‘Automatic Image Quality Assessment in Python | by Ricardo Ocampo | Towards Data Science Jessica Stillman Jeff Bezos Says the 1-Hour Rule Makes Him Smarter. New Neuroscience Says He’s Right Jeff Bezos’s morning routine has long included the one-hour rule. New neuroscience says yours probably should too. + Octs0 W1ATK @ 348 Ww ntps:towardsdatascience.com/automatic-image-qualiy-assessment-in-python-391a6ve62c11 sans