Visit https://ebookluna.

com to download the full version and

explore more ebooks

BIG DATA ANALYTICS: Introduction to Hadoop, Spark,

and Machine-Learning 1st Edition - eBook PDF

_____ Click the link below to download _____

https://ebookluna.com/download/big-data-analytics-
introduction-to-hadoop-spark-and-machine-learning-
ebook-pdf/

Explore and download more ebooks at ebookluna.com

Here are some recommended products that might interest you.
You can download now and explore!

Introduction to Algorithms for Data Mining and Machine

Learning 1st edition - eBook PDF

https://ebookluna.com/download/introduction-to-algorithms-for-data-
mining-and-machine-learning-ebook-pdf/

ebookluna.com

Advances in Subsurface Data Analytics: Traditional and

Physics-Based Machine Learning 1st Edition - eBook PDF

https://ebookluna.com/download/advances-in-subsurface-data-analytics-
traditional-and-physics-based-machine-learning-ebook-pdf/

ebookluna.com

(eBook PDF) Introduction to Machine Learning with Python:

A Guide for Data Scientists

https://ebookluna.com/product/ebook-pdf-introduction-to-machine-
learning-with-python-a-guide-for-data-scientists/

ebookluna.com

(eBook PDF) Introduction to Data Analytics for Accounting

https://ebookluna.com/product/ebook-pdf-introduction-to-data-
analytics-for-accounting/

ebookluna.com
Mobility Patterns, Big Data and Transport Analytics: Tools
and Applications for Modeling 1st Edition - eBook PDF

https://ebookluna.com/download/mobility-patterns-big-data-and-
transport-analytics-tools-and-applications-for-modeling-ebook-pdf/

ebookluna.com

(eBook PDF) Intro to Python for Computer Science and Data

Science: Learning to Program with AI, Big Data and The
Cloud
https://ebookluna.com/product/ebook-pdf-intro-to-python-for-computer-
science-and-data-science-learning-to-program-with-ai-big-data-and-the-
cloud/
ebookluna.com

(eBook PDF) Creating Value with Big Data Analytics: Making

Smarter Marketing Decisions

https://ebookluna.com/product/ebook-pdf-creating-value-with-big-data-
analytics-making-smarter-marketing-decisions/

ebookluna.com

Process Safety and Big Data 1st Edition - eBook PDF

https://ebookluna.com/download/process-safety-and-big-data-ebook-pdf/

ebookluna.com

Intelligent Systems and Learning Data Analytics in Online

Education: A volume in Intelligent Data-Centric Systems
1st edition - eBook PDF
https://ebookluna.com/download/intelligent-systems-and-learning-data-
analytics-in-online-education-a-volume-in-intelligent-data-centric-
systems-ebook-pdf/
ebookluna.com
 BIG DATA
ANALYTICS
 BIG DATA
ANALYTICS
Introduction to Hadoop, Spark, and Machine-Learning
 About the Authors
Dr Raj Kamal did his MSc at the age of 17, published his first research paper in a UK
journal at the age of 18, wrote his first programme in FORTRAN that ran at ICTl904, also
at the age of 18, and completed his PhD from Indian Institute of Technology Delhi at 22.
He was awarded a scholarship for postgraduate studies at IIT Delhi, research fellowships
of CSIR, and post-doctoral fellowships at Uppsala University, Uppsala, Sweden, (15
months) and ICTP, Trieste, Italy (3 months).
He has 51 years of research and 46 years of teaching experience, respectively. He has
so far successfullyguided 19 PhD scholars and is currently supervising 2 PhD scholars and
research students. He has published about 94 research papers in journals and 61
conference papers of both international and national repute. He is lovingly referred by a
few colleagues as the 'learning machine' and by a few others as a 'human dynamo!',
perhaps because of his constant drive for understanding emerging technologies and
passion for acquiring latest knowledge and its dissemination.
He has worked as a professor of seven subjects, viz. computer science, computer
science and engineering, information technology, electronics, electronics and
communication engineering, electrical education and physics. He has served as a faculty
member in DeviAhilyaVishwavidyalaya,Indore (25 years), Punjabi University, Patiala (17
years), Medi-CapsUniversity, Indore (3 years), KalasalingamUniversity,Tamil Nadu
(2 years), and Guru Nanak Engineering College,Andhra Pradesh, and King Mongkut's
Institute of Technology,Bangkok.
He is widely known for his books published by McGraw-HillIndia, namely, The Internet
of Things (2017), Embedded Systems (3rd Ed. 2014), Computer Architecture (Schaum series
adaptation), and Internet and Web Technologies (16th Reprint 2017). His Embedded Systems
book has international editions from McGraw-HillUSA, McGraw-HillSingapore, McGraw•
Hill China and McGraw-HillSouth Korea.
Currently, he is working as a senior professor at Prestige Institute of Engineering
Management and Research in Indore, MadhyaPradesh.
Preeti Saxena is working as an associate professor of computer science at School of
Computer Science & Information Technology in Devi Ahilya Vishwavidyalaya, Indore,
Madhya Pradesh. She has over 18 years of teaching experience at undergraduate and
postgraduate levels. She has three years of experience in a multinational software
company as a software engineer and programmer. She has published and presented 35
research papers in various international/national journals and conferences. She
completed her MCA degree with distinction in 1996 from National Institute of Technology
(earlier called MACT), Bhopal. She was awarded the 'Ramanujan GoldMedal - 2008' in her
MTech (Computer Science). She did her PhD in mobile computing. She is currently
supervising six research scholars in computer networks, mobile computing, data
analytics, augmented reality and Internet of Things (IoT). She has a passion for acquiring
knowledge on emerging technologies.
 BIG DATA
ANALYTICS
Introduction to Hadoop, Spark, and Machine-Learning

Raj Kamal
Senior Professor
Prestige Institute of Engineering Management and Research
Indore, Madhya Pradesh

Preeti Saxena
Associate Professor
Devi Ahilya Vishwavidyalaya
Indore, Madhya Pradesh

•
McGraw Hill Education(India) Private Limited
CHENNAI
McGraw Hill Education Offices
Chennai New York St Louis San Francisco Auckland Bogota Caracas
Kuala Lumpur Lisbon London Madrid Mexico City Milan Montreal
Sanjuan Santiago Singapore Sydney Tokyo Toronto
• McGraw Hill Education (India) Private Limited
Published by McGraw Hill Education (India) Private Limited
444/1, Sri Ekambara Naicker Industrial Estate, Alapakkam, Porur, Chennai 600
116
Big Data Analytics
Copyright © 2019 by McGrawHill Education (India) Private Limited.
No part of this publication may be reproduced or distributed in any form or by
any means, electronic, mechanical, photocopying, recording, or otherwise or
stored in a database or retrieval system without the prior written permission
of the publishers. The program listings (if any) may be entered, stored and
executed in a computer system, but they may not be reproduced for
publication.
This edition can be exported from India only by the publishers,
McGraw Hill Education (India) Private Limited. 1 2 3 4 5 6 7 8 9 D10307422 21
20 19 18
[I] :2 3 4 5 6 7 8 9 D l 03074 :2:2 :21 :20 ~ 18
Printed and bound in India.
Print-Book Edition
ISBN (13): 978-93-5316-496-6
ISBN (10): 93-5316-496-6
E-Book Edition
ISBN (13): 978-93-5316-497-3
ISBN (10): 93-5316-497-4
Managing Director: Lalit Singh
Senior Portfolio Manager: Hemant K Jha
Associate Portfolio Manager: Tushar Mishra
Senior Manager-Content Development & Production Services: Shalini Jha
Senior Content Developer: Vaishali Thapliyal
Content Developer: Malvika Shah
Production Head: Satinder S Baveja
Assistant Manager-Production: Anuj K Shriwastava
General Manager-Production: Rajender P Ghansela
Manager-Production: Reji Kumar

Information contained in this work has been obtained by McGraw Hill

Education (India), from sources believed to be reliable. However, neither
McGraw Hill Education (India) nor its authors guarantee the accuracy or
completeness of any information published herein, and neither McGraw Hill
Education (India) nor its authors shall be responsible for any errors,
omissions, or damages arising out of use of this information. This work is
published with the understanding that McGraw Hill Education (India) and
its authors are supplying information but are not attempting to render
engineering or other professional services. If such services are required, the
assistance of an appropriate professional should be sought.

Typeset at APS Compugraphics, 4G, PKT 2, Mayur Vihar Phase-III, Delhi 96, and
printed at
Cover Designer: Creative Designer
Cover Printer:
Visit us at: www .mheducation.co.in
Write to us at:[email protected]
CIN:U22200TN1970PTC111531
Toll Free Number: 1800 103 5875
Preface

Data analysis involves numerical and statistical analysis techniques, which have been
widely used in many fields such as sciences, biology, research, industry, business and
even sports, since the 1960s. The first author of this textbook, Raj Kamal, himself used
analytical techniques in the 1970s for obtaining solutions using matrix multiplication,
inversion, transpose, determinants, linear equations, simultaneous equations using
matrices and least square fitting for finding parameters from observed data points, which
can be described theoretically by superimposition of the functions. His first programme
was in FORTRAN that ran on ICT1904 in 1967. A classic book, Numerical Methods for
Scientists and Engineers, Richard W. Hamming, McGraw-Hill,New York, 1973 (Availableat
the ACM digital library), fuelled his interest in the field of analytics since then. Both the
authors learned from excellent lessons on Big Data Analytics and Advanced Big Data
Analytics given by Ching-Yung Lin, PhD and adjunct professor at Departments of
Electrical Engineering and Computer Science,ColumbiaUniversity USA, in 2017. It is here
that an idea of writing a textbook on Big Data Analytics for young minds came to the
authors.
The chess match of the legendary Garry Kasparov in 1997 against the IBM
supercomputer, Deep Blue, is a landmark moment in the history of computing
technology. "It was the dawn of a new era in artificial intelligence: a machine capable of
beating the reigning human champion at this most cerebral game", in the words of Garry
himself. Nowadays, data analytics, decisions, predictions and discovery of new
knowledge, are possible with the use of AI techniques of machine learning and deep
learning. The rise in technology has led to production and storage of voluminous amount
of data. Earlier, megabytes (106 B) were used and now petabytes (1015 B) plus are used for
processing, analysis, predicting, decisions, discovering facts and generating new
knowledge. Big Data storage, processing and analysis, face challenges from large growth
in volume of data, variety of data, various forms and formats, increasing complexity, fast
generation of data and the need to quickly process, analyse and use data.
Many applications such as industry reports, financial reports, social network and
social media, cloud applications, public and commercial websites, scientific experiments,
simulators, sensors in Internet of Things, and e-services generate Big Data. Big Data
Analytics (BDA) finds applications in many areas, such as healthcare, medicine,
advertising, marketing, sales, and tracing anomalies in big data in these disciplines
This textbook explains the concepts of BDA in a simple to complex manner. For
example, it uses the popular Ravensburger Beneath the SeaJigsaw Puzzle (5000pieces) in
an example to show that scaling out
and division of the computations along with data works well in parallel processing
shared-nothing architecture at distributed computing nodes. This student-friendly
textbook has a number of illustrations, sample codes, case studies and real-life analytics
for datasets such as toys, chocolates, cars, students' GPAs and academic performance.
Classic Apache-based Hadoop ecosystem tools and the latest Apache Spark ecosystem
tools deploying the Python libraries for analytics have been described in depth.

Readers
This textbook is an extremely useful asset for national as well as international students of
Big Data Analytics. This book caters to the needs of undergraduate and postgraduate
students of computer science and engineering, information technology, and related
disciplines, along with professionals in the industry for developing innovative Big Data
Analytics solutions based on Spark ecosystem tools with Python libraries, which include
the use of machine-learning concepts.
The book will also be a useful guide in training programmes for Big Data architects
and analytics requiring new skills, and for those who wish to learn the latest topics.
The main features of the book are:
• Easy-to-understand and student-friendly content, which includes
illustrative figures, examples and sample codes
• The book explains architecture, storage and programming methods for
Big Data analytics, while keeping multidisciplinary undergraduate and
postgraduate students as primary readers in mind
• Learning objectives for each section, recall from previous chapters and
introduction along with meanings of important key terms have been
provided in the beginning of each chapter
• Self-assessment questions, classified into three difficulty levels, have
been given at the end of each section in a chapter
• Key concepts covered, learning outcomes, objective questions, review
questions and practice exercises have been provided towards the end of
the textbook.

Roadmap for BDA Readers

The author presumes the readers possess basic intellectual and academic background in
mathematical and statistical methods, cloud platform for storage and applications (such
as Amazon 53), object oriented programming, familiarity and programme-writing skills in
Java, and the knowledge of Python libraries.
Readers intending to learn and use Hadoop ecosystem tools need to study chapters 1
to 4 and 6 to 9. They need prerequisite programme-writing skills in Java. Readers
intending to learn and use the latest Spark ecosystem tool need to study chapters 1, 3 and
5 to 10. They need familiarity and programme-writing skills in Java and Python libraries.
 Visit https://ebookluna.com
now to explore a diverse
collection of ebooks available
in formats like PDF, EPUB, and
MOBI, compatible with all
devices. Don’t miss the chance
to enjoy exciting offers and
quickly download high-quality
materials in just a few simple
steps!
Learningand Assessment Tools
Learning Objectives
Designof this book followsLearning Objective(LO) - oriented approach. This educational
process emphasises on developing required skills amongst the students. The process tests
the outcomes of the study of a course, as opposed to routine learning. This approach
creates an ability to acquire knowledge and apply fundamental principles to analytical
problems and applications.
Self Assessment Exercises
Each learning objective is followed by a set of questions for self-assessment. This offers
great retention of concepts.
Pedagogical Classification
The pedagogy is arranged as per levels of difficulty-all checkpoint problems are linked
with Learning Objectives(Los) and marked with Levelsof Difficulty(LOD), to help assess
students' learning. These levels of difficulty have been derived as per Bloom'staxonomy.
o o • indicates Level 1 and Level 2, i.e. knowledge and comprehension-based
easy-to-solve problems.
o • • indicates Level 3 and Level 4, i.e. application and analysis-based
medium to difficult problems.
• •• indicates Level 5 and Level, 6 i.e. synthesis and evaluation-based very
difficult problems.
Learning Outcomes
Summary points specific to each LO are provided at the end of each chapter. This
helps in recapitulating the ideas initiated with the outcomes achieved.
Chapter-end Exercises
More than 300 carefully designed chapter-end questions and exercises are arranged as
per levels of difficulty, and are framed to enhance knowledge and test new skills learnt.
These include objective type multiple choice questions, review questions and practice
exercises.

Salient Features
• Extensive coverage of topics in Big Data Analytics, such as Big Data
NoSQL Column-family, Object and Graph databases, Data reporting and
visualization, Programming with open source Big Data Hadoop
ecosystems tools, Spark, Spark ecosystem, Streaming, GraphX, and
 Mahout tools, have been explained using examples of datasets of interest
to students, such as toys, chocolates, cars and GPAs/academic
performance of students in theory and practical subjects.
• Latest topics such as Machine Learning, Regression analysis, K-NN,
Predictive analytics, Clustering, Decision trees, Clusters, and Similar,
frequent item sets, Pattern mining solutions, Classifiers, Recommenders,
Real-time streaming data analytics, Graph networks for web and social
network analytics, and Text analytics.
• Systematic approach: Data architecture is followed by Analytics
architectures, and the section on Hadoop ecosystem tools is followed by
Spark- and Python-based tools. Each chapter starts with learning
objectives and a quick recall from earlier chapters. The introduction is
followed by important key terms in the beginning of each chapter for
easy understanding of the chapter content. The text has been tagged
with descriptions and questions, self-assessment exercises and
illustrations within the chapter, and each chapter ends with learning
outcomes, MCQs, review questions and practice exercises.
• Rich pedagogy: 20+ programming codes, 100+ questions, solved
examples and practice exercises. Dedicated chapter on a major case
study in the textbook, and another major case study in online content.
Rich online content, PPTs, guide to solutions of practice exercises and
list of select books and references, which makes a comprehensive
bibliography for anyone interested in pursuing further studies in Big
Data Analytics.

Chapter Organization
Chapter 1 gives overview of Big Data, characteristics, types and classification methods. It
describes scalability, need of scaling up and scaling out of processing, analytics using
massively parallel processors, and cloud, grid and distributed computing. This chapter
introduces data architecture design, data management, data sources, data quality, data
pre-processing and export of pre-processed data stores to cloud. Approaches of
traditional systems, such as SQL, Relational Database Management System (RDBMS),
enterprise servers and data warehouse for data storage and analysis, as well as the
approaches for Big Data storage, processing and analytics have been explained in detail. It
also includes Berkley Data Analytics architecture, and introduces cases, case studies and
applications of BOA to its readers.
Chapter 2 starts with an interesting example, explaining the distributed parallel
computing architecture with shared-nothing architecture. This chapter describes basics
of Hadoop, its ecosystem components, streaming and pipe functions, Hadoop physical
architecture, Hadoop distributed file system (HDFS). It explains how to organize nodes for
computations using large-scale file systems, and provides a conceptual understanding of
MapReduceDaemon, functioning of Hadoop MapReduceframework, YARN for managing
resources along with the application tasks. The chapter introduces Hadoop ecosystem
interactions, and application support for analytics using AVRO, Zookeeper, Ambari,
HBase,Hive,Pig and Mahout.
Chapter3 highlights NoSQL data stores, solutions, schema-less models and increasing
flexibility of NoSQL for data manipulation. It describes NoSQL data architecture patterns,
namely the key value pairs, graphs, column family, tabular, document and object in the
data stores. This chapter explains the use of the shared-nothing architecture, choosing a
distribution model, master-slave versus peer-to-peer, and four ways which NoSQL
handles Big Data problems. The chapter covers MongoDB and Cassandra databases.
Chapter 4 describes the MapReduce paradigm, map tasks using key-value pairs,
grouping-by-keys and reduce tasks. It provides the conceptual understanding of
partitioning and combiners in the application execution framework, and MapReduce
algorithms by stating various examples. The chapter also describes Hive, HiveQL and Pig
architecture, Grunt shell commands, data model, Pig Latin. It provides an understanding
how to develop scripts and User-DefinedFunctions.
Chapter 5 introduces Spark architecture features, software stack components and
their functions. It describes the steps in data analysis with Spark, and usage of Spark with
Python advanced features. The highlight of this chapter is the description of methods of
downloading Spark, programming with the RDDs, usage of the Spark shell, developing
and testing Spark codes, and the applications of MLib. The chapter gives understanding of
how to run ETL processes using the built-in functions, operators and pipelines. It also
covers data analytics, data reporting and data visualization aspects.
Chapter 6 lucidly explains the classes of variables, and the ways of estimating the
relationships, outliers, variances, probability distributions, errors and correlations
between variables, items and entities. The chapter gives detailed descriptions of
regression analysis, and the use of K-NN distance measures for making predictions using
interpolations and extrapolations. It explains machine-learning methods of finding
similar items, similarities, filtering of simliars, frequent itemset mining, collaborative
filtering, associations and association rules mining. The highlight of this chapter is the
description of ML methods of clustering, classifiers and recommenders, and Apache
Mahout algorithms for big datasets.
Chapter 7 provides understanding of the concept, model, architecture, management
of data streams. It describes stream sources and stream computing aspects - sampling,
filtering, counting distinct elements, frequent itemset stream analytics, handling of large
datasets, and mining of association rules. The chapter explains the real-time analytics
platform, Apache SparkStreaming, and case studies on real-time sentiment analytics and
stock price analytics.
Chapter8 describes the modelling of databases as the graphs and representations of
graphs using triples. The highlight of this chapter is the description of use of graphs and
graph networks. The chapter gives methods of choosing the graph and graph parameters,
such as centralities for analytics. It explains the graph methods of diagnostics, decisions,
StatsModel, and probabilities-based analytics. Another highlight is the description of
features of Apache Spark GraphX,and its architecture, components and applications.
Chapter 9 describes text mining and the usage of ML techniques . Naive-Bayes
analysis, and support-vector machines (SVMs) for analysing text. The chapter explains
the methods of web mining, link analytics, analysing of web graphs, PageRank methods,
web structure analytics, finding hubs and communities, social-network analysis, and
representation of social networks as graphs. It describes computational methods of
finding the clustering in social network graphs, SimRank,counting triangles (cliques) and
discovering the communities.
Chapter 10 describes installation methods for Hadoop, Hive, Pig and Spark on the
Ubuntu platform. The highlight of this chapter is deploying and exploring open-source
Lego datasets, schema, processing and storage. The chapter explains MapReduce
implementation for counting items in a dataset, creating Hive data tables from a CSV
format dataset, and creating Dataframes from RDDs. It describes Hive and PySpark
programmes using functions for Merge and Join of Dataframes, the SQL-equivalentJoin,
and the UDFs for customised query processing. The chapter explains programmes for data
visualization using pi, bar and scatter plots. Another highlight of the chapter is the
description of machine learning programmes using sklearn for SVMs, Naive Bayes
Classifiers,linear and polynomial regression analyses, and predictive analytics.
Followingcontent is available towards the end of this textbook:
1. Solution to objective questions

2. Bibliography
• Printed and e-books
• Website resources
• Research journals
• Reference papers

Online LearningCenter
An accompanying web supplement available at http://www.mhhe.com/kamal/bda
includes:
PowerPoint slides for each chapter to supplement lecture presentations
Solution guide to practice exercises
Write-up on topics
An additional case study using an open source large dataset of car
company
Although much care has been taken to ensure an error-free text, a few mistakes may
have crept in. The authors shall be grateful if they are pointed out by the readers.
Feedback on content of the book as well as the web supplement available on the McGraw•
Hill site from readers will be highly appreciated through e-mail to
dr [email protected] and [email protected].
RAJ KAMAL
PREETISAXENA
Acknowledgements

Raj Kamal is grateful to Chairman, Dr N N Jain, Vice Chairman, Dr DavisJain, and Shri
Ketan Jain of Prestige Educational Society for providing him an opportunity to serve at
PIEMR, a great institution and platform to learn, and continue research and teaching in
new and exciting areas of engineering and technology. The author is also grateful to Dr
Manojkumar Deshpande,Director, PIEMR, for his continuous encouragement.
Raj Kamalis grateful to Sushil Mittal (wife)for her love and being a constant source of
support, and his family members, Dr Shilpi Kondaskar (daughter), Dr Atul Kondaskar
(son-in-law), Shalin Mittal (son) Needhi Mittal (daughter-in-law), and grandchildren
Arushi Kondaskar, Atharv Raj Mittal, Shruti Shreya Mittal and Ishita Kondaskar for their
love and affection.
Preeti Saxena extends her thanks to family members, Manish Saxena (husband) for
unconditional support and everlasting faith, Devansh Saxena (son) and Raghvi Saxena
(daughter) for their warmth and affection, Suvidya Saxena (mother) and Prateek Saxena
(brother) for their encouragement and well wishes.
Both the authors are also grateful to their colleagues, especially Dr Suresh Jain, Dr
(Mrs)Maya Ingle,
Dr Sanjay Tanwani, Dr Shraddha Masih, Dr Archana Chaudhary, Dr Manju
Chattopadhaya, Ms Kirti Panwar Bhati, Dr Savita Kolhe and Ms Pritika Bahad for their
support and continuous encouragement.
The authors would also like to thank the team at McGraw-HilE l ducation who initiated
the idea of writing, perhaps the first textbook on Big Data Analytics and whose agile
approach lead to the timely production of this textbook. The authors also thank the
reviewers who took out time to go through the manuscript and shared their valuable
suggestions.
Contents

Preface
Acknowledgements
List of Acronyms

1. Introductionto Big Data Analytics

1.1 Introduction
1.1.1 Need of Big Data
1.2 Big Data
1.2.1 Classification of Data-Structured, Semi-structured and
Unstructured
1.2.2 Big Data Definitions
1.2.3 Big Data Characteristics
1.2.4 Big Data Types
1.2.5 Big Data Classification
1.2.6 Big Data Handling Techniques
1.3 Scalability and Parallel Processing
1.3.1 Analytics Scalability to Big Data
1.3.2 Massively Parallel Processing Platforms
1.3.3 Cloud Computing
1.3.4 Grid and Cluster Computing
1.3.5 Volunteer Computing
1.4 Designing Data Architecture
1.4.1 Data Architecture Design
 1.4.2 Managing Data for Analysis
1.5 Data Sources, Quality, Pre-Processing and Storing
1.5.1 Data Sources
1.5.2 Data Quality
1.5.3 Data Pre-processing
1.5.4 Data Store Export to Cloud
1.6 Data Storage and Analysis
1.6.1 Data Storage and Management: Traditional Systems
1.6.2 Big Data Storage
1.6.3 Big Data Platform
1.6.4 Big Data Analytics
1.7 Big Data Analytics Applications and Case Studies
1.7.1 Big Data in Marketing and Sales
1.7.2 Big Data and Healthcare
1.7 .3 Big Data in Medicine
1.7.4 Big Data in Advertising
Key Concepts Learning
Outcomes Objective Type
Questions Review
Questions
Practice Exercises

2. Introductionto Hadoop
2.1 Introduction
2.2 Hadoop and its Ecosystem
2.2.1 Hadoop Core Components
 2.2.2 Features ofHadoop
2.2.3 Hadoop Ecosystem Components
2.2.4 Hadoop Streaming
2.2.5 Hadoop Pipes
2.3 Hadoop Distributed File System
2.3.1 HDFS Data Storage
2.3.2 HDFS Commands
2.4 Mapreduce Framework and Programming Model
2.4.1 Hadoop MapReduce Framework
2.4.2 MapReduce Programming Model
2.5 Hadoop Yarn
2.5.1 Hadoop 2 Execution Model
2.6 Hadoop Ecosystem Tools
2.6.1 Hadoop Ecosystem
2.6.2 Ambari
2.6.3 HBase
2.6.4 Hive
2.6.5 Pig
2.6.6 Mahout
Key Concepts Learning
Outcomes Objective Type
Questions Review
Questions
Practice Exercises

3. NoSQLBig Data Management,MongoDB and Cassandra

 3.1 Introduction
3.2 NoSQL Data Store
3.2.1 NoSQL
3.2.2 Schema-less Models
3.2.3 Increasing Flexibility for Data Manipulation
3.3 NoSQL Data Architecture Patterns
3.3.1 Key-Value Store
3.3.2 Document Store
3.3.3 Tabular Data
3.3.4 Object Data Store
3.3.5 Graph Database
3.3.6 Variations of NoSQL Architectural Patterns
3.4 NoSQL to Manage Big Data
3.4.1 Using NoSQL to Manage Big Data
3.5 Shared-Nothing Architecture for Big Data Tasks
3.5.1 Choosing the Distribution Models
3.5.2 Ways of Handling Big Data Problems
3.6 MongoDBDatabase
3. 7 Cassandra Databases
Key Concepts Learning
Outcomes Objective Type
Questions Review
Questions
Practice Exercises

4. MapReduce, Hive and Pig

 Visit https://ebookluna.com
now to explore a diverse
collection of ebooks available
in formats like PDF, EPUB, and
MOBI, compatible with all
devices. Don’t miss the chance
to enjoy exciting offers and
quickly download high-quality
materials in just a few simple
steps!
4.1 Introduction
4.2 MapReduce Map Tasks, Reduce Tasks and MapReduce Execution
4.2.1 Map-Tasks
4.2.2 Key-Value Pair
4.2.3 Grouping by Key
4.2.4 Partitioning
4.2.5 Combiners
4.2.6 Reduce Tasks
4.2.7 Details of MapReduce Processing Steps
4.2.8 Coping with Node Failures
4.3 Composing MapReduce for Calculations and Algorithms
4.3.1 Composing Map-Reduce for Calculations
4.3.2 Matrix-Vector Multiplication by MapReduce
4.3.3 Relational-Algebra Operations
4.3.4 Matrix Multiplication
4.4 Hive
4.4.1 Hive Architecture
4.4.2 Hive Installation
4.4.3 Comparison with RDBMS (Traditional Database)
4.4.4 Hive Data Types and File Formats
4.4.5 Hive Data Model
4.4.6 Hive Integration and Workflow Steps
4.4.7 Hive Built-in Functions
4.5 HiveQL
4.5.1 HiveQLData Definition Language (DDL)
4.5.2 HiveQLData Manipulation Language (DML)
 4.5.3 HiveQLFor Querying the Data
4.5.4 Aggregation
4.5.5 Join
4.5.6 Group by Clause
4.6 Pig
4.6.1 Apache Pig - Grunt Shell
4.6.2 Installing Pig
4.6.3 Pig Latin Data Model
4.6.4 Pig Latin and Developing Pig Latin Scripts
Key Concepts Learning
Outcomes Objective Type
Questions Review
Questions
Practice Exercises

5. Sparkand Big Data Analytics

5.1 Introduction
5.2 Spark
5.2.1 Introduction to Big Data Tool-Spark
5.3 Introduction to Data Analysis with Spark
5.3.1 Spark SQL
5.3.2 Using Python Advanced Features with Spark SQL
5.3.3 Data Analysis Operations
5.4 Downloading Spark, and Programming using RDDs and MLIB
5.4.1 Downloading, Installing Spark and Getting Started
5.4.2 Programming with RDDs
 5.4.3 Machine Learning with MLib
5.5 Data ETL (Extract, Transform and Load) Process
5.5.1 Composing Spark Program Steps for ETL
5.6 Introduction to Analytics, Reporting and Visualizing
5.6.1 Introduction to Analytics
5.6.2 Data/Information Reporting
5.6.3 Data Visualization
Key Concepts Learning
Outcomes Objective Type
Questions Review
Questions
Practice Exercises

6. Machine LearningAlgorithmsfor Big Data Analytics

6.1 Introduction
6.2 Estimating the Relationships, Outliers, Variances, Probability
Distributions and Correlations
6.2.1 Relationships-Using Graphs, Scatter Plots and Charts
6.2.2 Estimating the Relationships
6.2.3 Outliers
6.2.4 Variance
6.2.5 Probabilistic Distribution of Variables, Items or Entities
6.2.6 Correlation
6.3 Regression Analysis
6.3.1 Simple Linear Regression
6.3.2 Least Square Estimation
6.3.3 Multiple Regressions
 6.3.4 Modelling Possibilities using Regression
6.3.5 Predictions using Regression Analysis
6.3.6 K-Nearest-Neighbour Regression Analysis
6.4 Finding Similar Items, Similarity of Sets and Collaborative Filtering
6.4.1 Finding Similar Items
6.4.2 Jaccard Similarity of Sets
6.4.3 Collaborative Filtering as a Similar-Sets Finding Problem
6.4.4 Distance Measures for Finding Similar Items or Users
6.5 Frequent Itemsets and Association Rule Mining
6.5.1 Frequent Itemset Mining
6.5.2 Association Rule- Overview
6.5.3 Apriori Algorithm
6.5.4 Evaluation of Candidate Rules
6.5.5 Applications of Association Rules
6.6 Clustering Analysis
6.6.1 Overview of Clustering
6.6.2 K-Means
6.6.3 Hierarchical Clustering
6. 7 classification
6.7.1 Concept of Classification
6.7.2 K-Nearest Neighbour Classifier
6. 7 .3 Stochastic Gradient Descent Method - Logistic Regression
6. 7.4 Decision Tree Algorithm
6.7.5 Naive-Bayes Theorem - Naive Bayes Classifier
6.7.6 Support Vector Machine Classifier
6.7.7 Random Forest Classifier
 6. 7 .8 AdaBoost and Other Ensemble Classifiers
6.8 Recommendation System
6.8.1 Collaborative Recommendation
6.8.2 Model for Recommendation Systems
6.8.3 Content Based Recommendation
6.8.4 Knowledge-based Recommendation
6.8.5 Hybrid Recommendation Approaches
6.9 Apache Mahout Machine-Learning Applications
Key Concepts Learning
Outcomes Objective Type
Questions Review
Questions
Practice Exercises

7. Data Stream Mining and Real-Time Analytics Platform•

SparkStreaming
7 .1 Introduction
7 .2 Data Stream Concepts and Data Stream Management

7.2.1 Data Stream Concepts

7 .2.2 Data Stream Model
7.2.3 Architecture
7.2.4 Data Stream Management System (DSMS)
7 .2.5 Example of Sources of Streams
7 .2.6 Stream Queries
7 .2. 7 Stream Processing Issues
7.2.8 Real-time Processing, Stream Processing and Batch
Processing
 7.2.9 Summarizing Streaming Processing Needs
7.3 Stream Computing Aspects
7.3.1 Stream Computing
7.3.2 Sampling Data in a Stream
7.3.3 Filtering of Stream
7.3.4 Counting Distinct Elements in a Stream
7.3.5 Estimating Moments
7.3.6 Counting of l's in a Window
7.3.7 Decaying Windows
7.4 Frequent Itemsets
7.4.1 Finding Frequent Itemsets
7.4.2 Handling Large Datasets for Finding Frequent Itemsets
7.4.3 Limited Passes Algorithms
7.4.4 Counting Frequent Items in a Stream
7.5 Real-time Analytics Platform (RTAP)-SparkStreaming
7.5.1 Apache®Spark™ Streaming
7 .5.2 Real-Time Analytics Platform Applications
7.5.3 Case Studies-Real-Time Sentiment Analysis, Positive
Negative Sentiments Prediction and Stock Market
Predictions
Key Concepts Learning
Outcomes Objective Type
Questions Review
Questions
Practice Exercises

8. GraphAnalytics for Big Data and SparkGraphXPlatform

8.1 Introduction
8.2 Graph Model
8.2.1 Representing a Graph as Triples
8.2.2 Resource Description Framework (RDF) for Graph Databases
8.2.3 SPARQL Querying Language for RDF Graph-Database
8.2.4 NativeDBGraph Database
8.2.5 Property Graph Model
8.3 Graphs, Network Organization and Graph Analytics
8.3.1 Network Organization
8.3.2 Probabilistic Graphical Network Organizations-Bayesian and
Markov Networks
8.3.3 Graph Analytics
8.3.4 Choosing Graph Analytics
8.3.5 Use Cases of Graph Analytics
8.4 Graph Analytics Algorithms and Approaches
8.4.1 StatsModel and Probability Based Analytics
8.4.2 Technical Complexity in Analyzing Graphs
8.5 Spark GraphX Platform
8.5.1 Features of a Graph Analytics Platform-Apache
SparkGraphX
8.5.2 Dedicated Appliances for Graph
Key Concepts Learning
Outcomes Objective Type
Questions Review
Questions
Practice Exercises
9. Text, Web Content,Link, and Social Network Analytics
9 .1 Introduction

9.2 Text Mining

9.2.1 Text Mining
9.2.2 Naive Bayes Analysis
9.2.3 Support Vector Machines
9.3 Web Mining, Web Content and Web Usage Analytics
9.3.1 Web Mining
9.3.2 Web Content Mining
9.3.3 Web Usage Mining
9.4 Page Rank, Structure of Web and Analyzing a Web Graph
9.4.1 Page Rank Definition
9.4.2 Web Structure
9.4.3 Computation of PageRank and PageRank Iteration
9.4.4 Topic Sensitive PageRank and Link Spam
9.4.5 Hubs and Authorities
9.4.6 Web Communities
9 .4. 7 Limitations of Link, Rank and Web Graph Analysis
9.5 Social Networks as Graphs and Social Network Analytics
9.5.1 Social Network as Graphs
9.5.2 Social Graph Network Topological Analysis using Centralities
and PageRank
9.5.3 Social Graph Network Analysis using K-core and
Neighbourhood Metrics
9.5.4 Clustering in Social Network Graphs
9.5.5 SimRank
 9.5.6 Counting Triangles and Graph Matches
9.5.7 Using SparkGraph (Map-Reduce) for Network Graphs
9.5.8 Direct Discovery of Communities
Key Concepts Learning
Outcomes Objective Type
Questions Review
Questions
Practice Exercises

10. ProgrammingExamples in Analytics and Machine Learning using

Hadoop, Sparkand Python
10.1 Introduction
10.2 Installation Steps for Hadoop and Spark

10.2.1 Installation Steps for Hadoop, Hive and Pig

10.2.2 Installation Steps for the Spark on Ubuntu

10.2.3 Computing Platform Configuration

10.3 Datasets Used in the Examples, Data Deployment and Exploration

10.3.1 Counting and Sorting of Items in Datasets using MapReduce

10.3.2 Storing CSV Dataset into Hive Database

10.3.3 Storing CSV Dataset into the Spark DataFrame
10.3.4 Creating DataFrame from the RDD
10.4 Programming Steps using Hive and Pyspark

10.4.1 Merge and Join Functions for DataFrame Objects

10.4.2 Analysis and Query-Processing Using UDFs in Hive and

Pyspark
10.5 Data Visualization using Python Plotting Library
10.6 Machine-Learning Algorithms Implementation
 10.6.1 Clustering Algorithm
10.6.2 Classification Algorithm Example 1: SVM Classifier
10.6.3 Classification Algorithm Example 2: Naive Bayes Classifier

10.6.4 Regression Analysis Algorithms

Practice Exercises
Bibliography
Answers to Multiple Choice Questions
Index
 Visit https://ebookluna.com
now to explore a diverse
collection of ebooks available
in formats like PDF, EPUB, and
MOBI, compatible with all
devices. Don’t miss the chance
to enjoy exciting offers and
quickly download high-quality
materials in just a few simple
steps!
Other documents randomly have
different content
 In other states, for Buffalo substitute Cleveland, or Chicago or
Milwaukee—great cities unequally yoked with an agricultural
population of divergent interests.
It is inevitable under the circumstances that the states through
their legislatures should do a good deal of polite nullifying on their
own account. The provisions of the constitutions relating to
legislative powers over counties, instead of being strictly construed
have ingeniously circumvented. What some legislatures could do in
defiance of good political science but yet without legal evasion the
California, New Jersey and other legislatures have accomplished by
stretching the meaning “general” or “uniform.”
Take the California practice. In the words of the constitution the
legislature is required to establish “a system of county government
which shall be uniform throughout the state.” But it happens that
these counties range in population from a few hundred to over half a
million, and in area from 755 to 23,000 square miles. Some are
strictly rural, while, at the other extreme, is one geographically
identical with the city of San Francisco. All sorts of combinations of
urban and rural conditions intervene. Some of the territory is
traversed by steam railroads and trolley lines and some of it is
inaccessible to a stage coach. But all of it is “uniformly” governed.
Inasmuch as the legislature never could bring itself to withhold its
hand from the minute details of county business, it had to find a way
to “beat” the constitution. It placed each of the fifty-six counties in a
separate class, and passed fifty-six “general” laws, each applying in
fact to a single county, but not mentioning the county by name!
In Illinois the habit of special legislation has led the Bureau of
Public Efficiency to remark that: “The General Assembly of Illinois
might with propriety be added to the list of nineteen local governing
bodies of Cook County, for it is continually interfering in an arbitrary
manner in matters of local administration.”
New Jersey has sinned quite as grievously and its courts have
consistently upheld the act even against a provision of the
constitution which expressly prohibits the legislature from
“regulating the internal affairs of towns and counties.”
But lest the full import of these statements should be lost, the
following titles of special county bills in a single session of a New
York legislature, are cited in evidence:

Authorizing conveyance of land on Holland Avenue.

Striking out the provision authorizing county treasurer to
appoint an attorney.
Regulating tax collection procedure.
Fixing compensation of unskilled laborers.
Correcting 1915 tax roll.
Creating a county auditor.
Increasing salary of sheriff, etc.
Levy of taxes to meet cost of sanitary trunk sewer.
Regulating management of penitentiary and workhouse.
Creating commissioner of charities.

Too often the motive of the legislators has not been to make the
county the state’s more obedient servant, but to “bleed” it to the
utmost for political purposes. Back of the real difficulties of adjusting
the state’s responsibilities to the idea of local control over
administrative details, is too evident the suspicion that the political
machine needs the county very much “in its business.”
For this reason, doubtless, special legislation affecting counties is
so often inseparably associated with the forcible opening of the
county treasury. New York City in recent years has suffered
grievously from mandatory salary increases, imposed in many cases
by a party of the opposite political faith from the one in control over
the local budget. Thus in 1915 out of a total budget allowance of
$7,003,716.82 for county purposes, the sum of $4,858,773.47 or
69.1 per cent. represented mandatory appropriations which could
not be increased or diminished by the local budget makers either
because the exact amount was fixed by law or because the power of
fixation was conferred upon other officers than the appropriating
body of the city. Many of the measures in question dealt with the
salaries of clerks, stenographers and messengers. Of the total
allowance in the same year for personal services (salaries, etc.) of
$5,809,481.75, 78 per cent. or $4,576,985.75 was beyond local
control. While by far the greater proportion of these sums were just
and necessary, the margin of waste which represented one hundred
per cent. politics, was, without a doubt, exceedingly large.
Nor have the legislatures been led to take this course for reasons
of public economy. In thus appropriating other people’s money
according to its professedly superior knowledge of the state’s needs,
it does not often appear to give heed to standards of experience or
of service rendered. Often it is but the old story of the influences
back of the bill with this title: “An Act, providing for the appointment
by the sheriff of —— County, of an undersheriff, fireman and court
officers, and for their compensation and duties.” When the truth
came out it appeared that the sheriff and the board of supervisors
were of opposite parties. The sheriff (whether rightly or wrongly)
was not to be put aside. He simply appealed over the heads of his
“superiors” to a higher authority for what he wanted—and got it.
The people had elected a Board of Supervisors, to manage the
county finances, but at the moment when this body might have
effected a just economy, it was forcibly stripped of their powers.
Theoretically the legislature in the case cited intervened, but what
probably happened was that someone saw the representative in the
legislature from the county in question and he “fixed” it with the
proper committee. The speaker let the bill go through on the floor of
the legislature.
From the standpoint of the local politician or petty officeholder
who is looking for special privilege, via the back-door method, and of
the home legislator who does the “fixing,” special legislation is thus
doubtless a benevolent privilege which enables men to put various
local people in their debt for future purposes. For the “organization”
of the dominant party, the power to give to or withhold from the
local municipalities is often not the least important element of its
power.
As for the county, for its sins of nullification it would seem to be
appropriately penalized, particularly if it belongs in the metropolitan
class. It has been placed under a sort of patriarchal discipline,
robbed of much of its individuality and initiative; its responsible
officers subjected to humiliation from subordinates; its resources
diverted to partisan uses.
 CHAPTER XIII
STATE GUIDANCE
At this point the indictment of the county ceases. It is not an altogether hopeless situation.
The very thoroughness of the county’s failure is the chief promise of ultimate redemption.
Not because the county is constructed on an unsound political theory, not because it has
shocked the sense of humanity, but because of its riotous misuse of public funds, it has begun
to attract the attention of higher authorities.
Where does the county’s money go? It has been strongly intimated in previous chapters that
the citizens of the county and sometimes even the county officers know little and care less. Is
it economically run? No one can easily tell, without knowing what other county governments
are costing, service for service and unit for unit. And no one can make such a comparison
between counties unless they have some common basis of understanding. To establish
standards in the use of terms, to make in other words, each county tell its financial story in a
language understood throughout the state, to bring the information from the various counties
together for comparison, to insist upon a sufficiently detailed description of financial activities,
is the object of uniform reporting.
And who can force such a coming together for a common understanding other than the
state itself?
Among the states where county government is of appreciable importance, Ohio was the
pioneer in the direction indicated by this suggestion. The law enacted in that state in 1902
approached the county problem with the conviction that what was needed above all else was
more light—in an administrative sense; that when the shortcomings of county government
could be reduced to statistics and comparisons (invidious if necessary) could be made between
various units, then some real improvements might be reasonably expected. Provision was
made in the enactment for a state Bureau of Inspection and Supervision of Public Offices
which should install a uniform system of public accounting, auditing and reporting in every
office in the state. A corps of field agents known as state examiners were employed on a civil
service basis to make personal examinations in each of the taxing districts. The findings of the
examiners are published, and if money is due the county the enforcement of the law is left first
to the county prosecuting attorney and then to the attorney general.
New York followed the lead of Ohio by passing in 1905 a law which requires counties,
villages and cities, to report annually to the comptroller on forms prescribed by him.
“Indiana and Ohio,” says Professor John A. Boyle,[14] “has gone into the science and art of
uniform accounting very seriously and very effectively. The Indiana law (1909, ch. 55,
amended March 3, 1911), creates a Department of Inspection and Supervision of Public Offices
having jurisdiction over every public office in the state. The administration of the law was
entrusted at the outset to one state examiner, two deputies, one clerk, and fifty-two field
agents working on a civil service basis. Uniform accounting is prescribed and installed.
Comparative statistics are compiled by the state examiner and published annually, so that the
fruits of this department are available to the public.”
Wyoming has a fair system of audit.
 The North Dakota law, while it requires the state examiner “to prescribe and enforce correct
methods,” does not call for a uniform system. Massachusetts, Kansas, Georgia, Iowa, Nevada,
Florida, Tennessee, New Mexico, Arizona, Colorado, Oklahoma, Washington, Minnesota, West
Virginia, Louisiana, California and Michigan have more or less complete systems of state
financial supervision. The “black sheep” among the states in this respect are Alabama,
Arkansas, Delaware, Illinois, Kentucky, Maine, Maryland, Mississippi, Missouri, New Hampshire,
North Carolina, Rhode Island, South Carolina, Texas, Utah, Vermont and Virginia.
Now for the results of this supervision.
Professor Boyle has summed up the sort of assistance that the state bureaus of accounting
have been able to give. One instance of such help is that where county officers had been
accustomed in the past to take long and expensive junkets to inspect public buildings, expert
advice under the new system has been rendered to them in much cheaper form through the
investigators of the state. Examiners have also been able to point out to county officers many
deviations from the letter of the law, the strict compliance with which is of the most vital
importance in the performance of certain county functions, such as taxation. In a similar way
they have checked up illegal charges against the county, inadequate audit (or no audit at all),
instances of additional compensation (under various guises) for personal service, illegal
temporary loans and misapplication of funds.
It will at once be seen that the mere possibility of a state examiner’s visit will have an
admonitory effect which in itself will often be sufficient to keep an official in the straight and
narrow path. The county, in conforming to the reporting requirements, derives a local benefit
wholly apart from any obligation to the state. Upon the basis of a sound and permanent
system of accounting the local officers are in a position accurately to inform the county of their
doings and make comparison of a financial transaction of one year with those of previous
years. Herein is one foundation stone of a scientific budget.
Under a complete system of state regulation not only are the forms and standards of
uniform accounting established, but a staff of expert examiners is created to determine by
periodical investigation, whether or not these standards are lived up to.
The remarkable conditions preceding the establishment of the Ohio Bureau and the
important services which it has rendered the state, are revealed in the following summary[15]
of its findings in counties during the first ten years of its existence.
STATEMENT OF FINDINGS TO NOVEMBER 15, 1912
COUNTIES

Findings
Illegal Unclaimed Total
Year for Returns
Payments Moneys Illegal
Recovery
1903 $50,268.93 $18,808.91 $807.92 $69,884.76 $10,741.93
1904 57,805.54 2,504.41 10,339.95 70,649.90 2,222.31
1905 246,280.58 7,421.87 25,389.52 279,091.97 24,847.83
1906 295,082.80 14,227.52 5,218.21 314,528.53 232,156.78
1907 646,397.50 115,906.91 18,049.13 780,353.54 322,911.08
1908 103,764.26 43,333.31 9,829.15 156,928.92 41,171.53
1909 410,282.51 320,137.17 23,219.42 753,639.10 66,219.91
1910 146,024.04 106,410.00 22,241.18 274,675.22 24,438.36
1911 233,547.24 129,007.02 7,921.90 370,476.16 37,735.34
1912 112,926.80 No report 118.26 113,045.06 96,015.16
Ttls $2,302,379.30 $757,759.32 $123,134.54 $3,183,273.16 $858,460.23

TAX ADMINISTRATION
A branch of local fiscal administration which is in far less satisfactory shape is that of
taxation. In no department is “nullification,” as has been already shown, more constant and
serious. The situation is complicated. In seventeen of the states, including every one north of
the Ohio and Potomac rivers, and east of the Mississippi, except Illinois, Indiana and Maryland,
the county as a whole has very little to do with assessments for the general property tax, the
unit for assessment in that section being the town or township. That in nearly all these states
the general property tax may be said to have broken down, would seem to be indicated by the
establishment of a permanent state tax commission or commissioner in all of them except
Pennsylvania. In thirty-one states in the South and West, the county is the unit of assessment,
and in its favor it may be said that in this rôle it has proven a far more satisfactory performer
than the town. The county is small enough to serve as a convenient unit for assessment
operations and assessment records and furnishes the basis of a system of fewer units than if
the town were the basis. Under the county plan, moreover, there are fewer opportunities for
communities to compete against each other in their effort to escape their just share of the
general burden of government. It is significant that most of the western states have seen the
advantages of the county as the local unit of administration.
But the county, for all that, appears to be incapable of standing on its own feet in tax
matters. Even under the most favorable circumstances there remains an important duty for a
state commission to supervise the work of the county assessor or board of assessors to the
end that the letter of the law may be obeyed with the utmost uniformity throughout the state.
Such a commission must see that the county does injustice to neither individuals nor the state
through inequal assessment and that the tax sales are in accordance with the law.
The county, in short, has a useful place in the general scheme of tax administration, but it
must be a supervised unit.

CIVIL SERVICE
In the administration of the civil service law the county also does well to lean upon the state.
The same considerations that apply in accounting and tax matters apply with equal force in the
selection of the employees. Most of the counties are too small to serve as units in which to
install facilities for conducting examinations and publishing useful records. The superior powers
of the state commission over the localities in these respects are emphatically not a destructive
check upon the county’s officers. They do not detract from local responsibility. They simply
enable them to apply to their work the most effective means available.
 In New York the state civil service commission regulates the service in eighteen counties. In
New Jersey the adoption of the civil service law involving state control rests with the people of
each county. Hudson, Essex, Mercer, Passaic and Union counties have taken advantage of the
law.
Administration apart from fiscal supervision in other departments, such as charities, prisons
and public health, has advanced much more slowly than reform in the fields which have been
mentioned. But every indication for the future is toward greater control through accounting
and supervision. The county acts locally in the enforcement of state obligations. The real
trouble comes when one undertakes, arbitrarily, to place any given county activity in the local
or state category. The catching of a thief, for example, is a very essential part of the state’s
most fundamental duty to protect property, but the locality where the crime is committed is
very keenly interested in having the machinery of the county set to work to punish the deed.
The locality pays the sheriff his salary or his fees, but the state, in protecting the property of
its citizens, is probably justified in guarding against extravagant or dishonest use by the people
of the county even of their own money.
And so it is impracticable and not by any means necessary to give the county quite unbridled
liberty to control all its officers. Central supervision is a middle course that protects the state
and instead of impairing, really conserves the county’s interests. The state, without instructing
the county as to what it may do with its purely local powers, may lay a firm hand upon it in
telling how to use those powers effectively. Such a course is conceived in quite a different spirit
from the destructive sort of state meddling which was described in the preceding chapter,
which proceeds from the legislative branch of the central government. What the state
government actually does when it regulates or supervises is to hold up the county official to
standards of performance. This is the proper work of officers or boards in the administrative
branch. It relates not to what the county shall do as a matter of broad policy, but how the
county shall redeem its obligation in matters of routine, detail and technique. The state agency
may have only the power to “visit and inspect,” like the New York State Board of Charities, or it
may, like the Comptroller of the same state, actually impose forms of procedure. In either
case, local officers otherwise unstimulated to efficient work, as they must be under the present
system of government, are thus given moral support from a responsible quarter. They are
subjected to a cold, unescapable comparison with other officers in other localities and they are
given the benefit of expert advice on many obscure and complex phases of public
administration.
[14] See Annals of the American Academy of Political and Social Science, May 1913. pp.
199-213.
[15] Prof. John H. Boyle, op. cit., p. 203.
 CHAPTER XIV
READJUSTMENTS
So much by way of accepting counties as they are. “State
guidance” goes a long way as a palliative of unsatisfactory
conditions. It is a sort of permanent first aid to the injured.
But the county needs surgical treatment! In some cases it is well
to fix responsibility. In extremities it becomes necessary to
amputate.
Bear in mind, to begin with, the fact that the county at bottom is
really a piece of the state, a local agency. The prevailing practice of
local election of officers and its logical sequence, local nullification,
have done much to obscure the real interests of the state at the
county court house, till the average run of citizens have long
forgotten that the distinction exists; that officers like the sheriff,
district attorney, public administrator and coroner are not strictly
local officers at all but subordinates of the general state government.
By far the most important branch of general administration with
which county officers serve is the judiciary. Counties, except in a few
states, are the units for selection of judges having original
jurisdiction in both civil and criminal cases which involve moderate
amounts of money and less than the most serious offenses. With the
county court also is generally associated probate jurisdiction, which
is exercised in some of the eastern states by a special officer known
as the surrogate, who may be a judge as in New York, or a purely
ministerial officer, as in New Jersey.
Legally the judiciary is more nearly a part of a state system than
any other branch of the county organization. The decisions of the
judges are of course subject to appeal to a higher state court—that
is an important form of control. Sometimes, as in California, a part of
the salary of the county judge is paid from state funds. Probably too
the greater popular respect that hedges about the bench is sufficient
to set it apart from much of the sinister influence that often affects
the other officers of the county. Nevertheless, the county court in
common with other divisions of the judiciary, is subject to a wide
variety of disintegrating forces. It has a variegated allegiance: to the
people (in most of the states) for its original selection, to the county
governing body for many incidental items of financial support, and to
higher judicial authority for confirmation or revision of its decisions.
It does not control its executive agents, such as the sheriff and the
county clerk, who are usually independent elective officers.
The readjustment of this situation can hardly be effected
satisfactorily apart from a complete reorganization of the state’s
judicial system. This will undoubtedly involve, among other things, a
much more complete central control on the part of a state chief
justice and a judicial council. The courts must be organized with a
keener appreciation that a judge in rendering just and learned
decisions is a part of a business machine—he is waiting on
customers; that there is no sound reason why the judicial
department should defy the principle of responsibility any more than
a department which serves the public in another way.
In the general overhauling of the systems, perhaps the elective
county judge will disappear. But liberty, for all that, will not vanish
from the earth. States as diverse in their location and composition as
New Hampshire, Connecticut, New Jersey and South Carolina never
drifted quite completely into the habit of “electing everybody,” and
they are good states. The Federal judiciary, too, is appointive, and it
is not more culpable, by standards of either the progressiveness or
of administrative efficiency, than most of the state courts.
The earthquake, we may hope, will also shake up the justices of
the peace, who in any self-respecting organization of the courts will
either disappear or be linked up, as the American Judicature Society
proposes, in a county system, with the county judge in control. The
justices are now associated with the town or a corresponding
division of the county and deal with very minor (but not for that
reason insignificant) civil and criminal cases, or act as a tribunal for
preliminary hearings and commitments. In cities the office has
steadily and hopelessly decayed by transfer of its jurisdiction to
other courts and through the abuses of the fee system. It was
established moreover when means of transportation were few and
difficult and districts consequently had to be made small in order to
meet the convenience of the litigants who came seeking justice. But
times have changed! Circumstances favor larger districts and
infinitely better control over this branch of the judiciary.
In making over the courts we cannot properly overlook the
machinery for enforcing judicial decisions. This is the function of the
sheriff. Where, in the reconstructed scheme of things shall he come
in? Our forefathers committed themselves to the theory of the
separation of powers, these three: the legislative, the executive and
the judicial. Under this scheme of things the duty of judges is to find
or interpret the law: nothing more. The sheriff as an executive
officer is therefore always independent of the court: he is the
enforcer of state laws which come to him in the form of very specific
instructions by way of court decisions. Such instructions are issued in
the interest of parties to a legal controversy.
But the sheriff is not the only law enforcer in the county. The state
has made the board of supervisors (or board of commissioners) and
county officers its agents in enforcing various state laws. And so the
question arises: why not bring all the enforcing agencies under a
single control? Two ways suggest themselves. If the judicial system
is to be a unified state affair, then judicial-decision enforcing should
also be a state concern and the county should keep its hands off. In
practice this would mean that the governor or some other general
state officer should appoint the sheriffs on the same principle which
is employed in the federal government, wherein the President
appoints U. S. marshals. But, if on the other hand, the county
government is to be considered as a general local agency of the
state for enforcing all its laws, then nothing remains but to put it up
to the local governing body or some local chief executive to select
the sheriff. Such is the method which has been applied in the city
and county of Denver, Colorado, where the mayor is the chief
executive of the consolidated governments.
And what of the coroner? Every authority worthy of credence is
agreed that this office, above all others in the county, has long
outlived its usefulness. That one small head should contain the
necessary skill of criminal investigator, medical expert and magistrate
is far too much to expect of any ordinary mortal. A few states,
following Massachusetts, which abolished the coronership in 1877,
have created an office under various titles, such as “medical
examiner,” “county physician” (New Jersey), “medical referee” (New
Hampshire), etc. They have modernized the coronership by stripping
it of its magisterial powers and taking it off the ballot. In most cases
the new medical officer is an expert pathologist and his services are
often of the greatest value in criminal and civil actions and to the
cause of science. In New York City the coroners now in office in the
five boroughs will go out of office on January 1, 1918, their powers
of investigation will then be transferred to a chief medical examiner
appointed by the mayor, and a corps of assistants who will be
equipped with ample laboratory facilities. The judicial duties of the
coroner will be turned over to the city magistrates.
As for the district attorney,[16] his proper relations to a criminal
trial and to the public seem to be generally misunderstood. He is the
state’s advocate as against the breaker of the law. But it is no part of
his business to send every alleged offender to the penitentiary. The
efficiency of the prosecutor is not to be determined by a high record
of convictions. He is not, properly, a man-hunter. Nor is it his
province to decide when he shall bear down hard upon offenders
and when he shall soften justice. His functions, in short, are
administrative and not political. And when that fact is admitted every
reason why he should be popularly elected falls. In the ideal county,
efficiency in the administration of justice will not be a perennial local
campaign issue, for the prosecutor will be appointed by some
responsible state authority, such as the governor or attorney
general, not as a reward for political services but on the basis of
merit and fitness. Or if, perchance, it shall be deemed unwise so to
centralize authority, it would at least be logical to let the county
authorities do the appointing.
There remains to be disposed of the clerk of court.[17] His relation
to the bench is rather a closer one than that of the sheriff, so close
that in some jurisdictions it has not been thought a violation of the
theory of the separation of powers to allow the judge or the whole
court to appoint him. Such indeed would seem the proper course.
But in many counties the business of court-clerking is hardly onerous
enough to engage a separate officer and the duties have accordingly
been transferred to the county clerk. This officer usually performs a
variety of functions, among which are his services as clerk of the
county board and as the local custodian or register of legal papers
required to be filed under certain state laws. In counties where this
“bunching” of functions has to be resorted to, the least that should
be done by way of readjustment should be to help along the
unification of the county government by vesting the appointment of
the county clerk in the county board or a county executive to be
established. In the larger counties where the volume of county
business warrants a separation of functions, there seems to be no
sound reason why such duties as the filing of papers should not be
in the hands of an officer appointed by some state official, to
represent him in the locality. In this way certain counties at least
would be completely divested of responsibilities of which they
appear never to have acquitted themselves too well. The county’s
ultimate place in the sun is being determined by a stripping process:
the state is taking up its work.
Thus in the domain of charities. We pointed out in Chapter IX that
the county was hopelessly deficient in caring for the insane, the
defectives and the criminals. Modern methods, which are humane
methods, have come to demand strict classification as the very
starting point for treatment. But unless the number of subjects for
treatment is reasonably large, the expense of such classification is
prohibitive. Most counties cannot supply the numbers. They find
themselves in the predicament of a rural sheriff who has in custody
an average of perhaps six prisoners. If his county were to obey the
law it would have ready at all times an establishment which provided
special accommodations for almost forty different classes of inmates.
Under these circumstances the only course is to transfer as many
classes of prisoners as possible to the care of a larger unit of
government that is able economically to segregate. This transfer
actually began in some of the older states in the eighteenth century.
Massachusetts led the movement by erecting a special prison on
Castle Island for the most desperate type of convicts. In 1796, New
York began the construction of two state prisons in New York and
Albany. In 1816, Ohio built a state penitentiary at Columbus and in
1839, Michigan completed its first state prison at Jackson. Reform
institutions in this country began to be established in Massachusetts
in 1846 when juvenile offenders were removed from local jails and
state prisons. Since then separate institutions for boys and girls have
been established in nearly all of the eastern states. New York in
1877 erected the first reformatory to which adult convicts could be
committed under an indeterminate sentence.
As for the care of the insane, this was the first department of
public welfare administration that was taken over by the whole state,
beginning with the establishment of the first hospital in Utica, New
York, in 1843. From time to time other states have followed New
York’s example until nearly every state has one or more hospitals.
With the increase in the number of institutions in a given state
further segregation and classification of inmates has been possible.
State institutions now furnish the means for appropriate education
for the mentally defective who formerly were left to shift for
themselves in mismanaged county almshouses. The deaf, dumb and
blind have been taken care of in similar fashion. Indeed, the function
of county poor relief would now seem to approach as its ideal, the
complete transfer to the state of all charity functions except possibly
a certain amount of temporary “out-door” relief. But even this rather
narrow field has been invaded, for, in New England and New York, at
least, a class of “state poor” is known to the statutes.
That the county has often sadly broken down in the guardianship
of life and property is a fact which has come into prominence within
very recent years. As the police force of the big cities become more
and more efficient the field of operation for criminals is transferred
to the suburbs, to the small towns and villages and to the open
country and the police problem in the rural sections takes on a semi-
metropolitan aspect. Good roads, the automobile and the telephone
have facilitated the business of thugs and burglars as well as of
honest citizens. Said the district attorney of Niagara County, New
York, recently, “Nearly every post office safe in Western New York
has been robbed, and I do not now recall anybody having been
convicted for these crimes. The ordinary constable or deputy sheriff
can serve subpœnas and make a levy under an execution providing
he is feeling well, but as a general rule he is incapable of coping with
even a third-class criminal.” Numerous other equally forcible official
statements of the same tenor have been collected by the New York
Committee on State Police.
To cope with these crimes of violence and cunning the untrained,
politically selected sheriff of the typical rural county is but sadly
equipped. He is a temporary elected official to begin with,
unschooled in the ways of criminals and unfamiliar with any of the
vast paraphernalia of investigation that go to make up a modern
police system. Certain parts of the country moreover have peculiar
periodical disturbances on a larger scale than criminals operate—
riots, lynching parties, flood disasters, strikes. These occasions
demand the temporary mobilizing of a comparatively large, well-
organized and disciplined force to handle the situation with firmness
and fairness. It is no place for the crude old-fashioned sheriff’s
posse.
At the present time it is the frequent practice on such occasions to
call out the state militia. But while this organization has often
doubtless rendered effective service, police duty is not its proper
occupation. Every year enlistments fall far below the adequate figure
because young men in business and professional life deem it
obnoxious to leave their appointed tasks to do a professional
policeman’s work. In the discussion of “preparedness” measures it
has frequently been proposed that the militia be enlisted solely for
national defense and that a special fighting force be developed for
state police duty. If such a force were organized on a permanent
basis it would practically relieve the sheriff and the constables from
police duty.
A model for such a force is found in the Pennsylvania State
Constabulary, which has been in existence since 1902. This is an
organization of two hundred and twenty mounted policemen formed
into four companies under a superintendent of police. Every year it
patrols 660,000 miles of rural roads and not only keeps the rural
sections singularly free from criminals but has performed numerous
other distinctive services. It has prevented disastrous fires and mine
explosions, quelled riots, stopped illegal hunting and maintained
quarantine during epidemics of disease. It was this constabulary that
handled the tremendous crowds at the Gettysburg Centennial in
1913. The force is made up of picked men who are taught the laws
of the commonwealth and schooled to enforce them with absolute
impartiality against offenders of all classes. It has been free from
politics and has won the respect of all classes of the people.
In the domain of highways[18] the county, under the pressure of
the good-roads movement, has been rapidly yielding its control to
the central government. The good-roads problem simply outgrew
the county. It could not be handled efficiently through so small a
unit. In the course of railway development everywhere the old lines
of tributary traffic by wagon road from the farms to the shipping
centers were greatly modified. Their objective point came to have no
particular reference to the boundaries of the county or the location
of the county seat. Traffic from one county destroyed the roads of
another without supplying any compensatory advantages to the
latter. Modern road construction, particularly since the advent of the
automobile, created technical engineering problems far beyond the
capacity of the local officials to solve. Without the aid of better
equipped agencies than a unit so small as the county could afford
most of the rural roads of the county must have gone to rack and
ruin, to say nothing of their meeting the demands of present day
traffic. But forty-two state governments, up to 1915, had come to
the rescue, either by supplying financial aid, authorizing the
employment of convict labor or by furnishing expert advice founded
upon scientific research. Up to the year 1914 only Florida, Indiana,
Mississippi, South Carolina, Tennessee and Texas had made no
provision whatever for state participation in road work.
But the significant point to be noted here is the strong tendency to
take entirely out of the hands of the county the whole burden,
financial and otherwise, of the great trunk lines and in many cases
to impose standards and specifications for construction even where
the county does its own road building. Thus, Massachusetts, up to
January 1, 1914, had completed more than one thousand miles of
state highway through the issuance of state bonds and the levying
of automobile taxes, the counties being required to refund the state
twenty-five per cent. of the cost of construction. New York
established a highway department in 1898 and has authorized bond
issues of $100,000,000 for a state system of roads which has
already reached an advanced stage of construction. Virginia, Ohio,
Maryland and California have made much progress toward a state
road system, the California plan calling for two main highways
running the length of the state and a system of laterals connecting
the county seats with the trunk lines. The state of Iowa has gone so
far as to place all road work in the state under the direction of its
highway department.
And so, the dominion of the county is being invaded at sundry
points. The unification of the judiciary (which it must be admitted
has not yet progressed very far), the gradual transfer of the charity
and correctional work to the state government, the establishment of
a state police and the more imminent abridgment of county control
over highways—these movements unmistakably and definitely seem
to point the ultimate displacement of the county as an important
agent of public service in particular fields.
But that is but one side of the story. For while, on the one hand,
the importance of the county is threatened in particular fields there
seems to be before it in other directions a career of greater
usefulness than ever before. This present observation, however,
applies only to those states where the town or township has come in
for particular emphasis, or where, as will be suggested later in the
discussion, the principle of federation may be adopted by a number
of contiguous municipalities as a step toward consolidation of local
governments.
In a number of states where the New England influence has been
strong the town is frequently the unit (though not always
exclusively) for the custodianship of certain records such as deeds
and mortgages, for public health administration, for commitment of
paupers, for road construction and maintenance or for tax
assessment and collection.
The relation of the county to the town in these concerns is
analogous to that of the state to county in such matters as the care
of the insane and the control of trunk-line highways. It is a question
of finding a unit large enough (and not too large) to fit the problem
in hand. Public health, for example, is largely a matter of controlling
sources of disease in milk and water supply, which under modern
methods of living are usually much more widely distributed than the
area which is served. Effective control in that case would simply
mean control through a unit larger than the town, to wit: the county,
unless control on a still wider scale should prove feasible. Similarly in
the matter of police protection: the town constable is an
anachronism in these days of rapid transit. The county is a more
appropriate police unit than the town. Town custodianship of records
means duplication, lack of standards and waste. Town commitment
of paupers to the county almshouse or poor farm is a temptation on
the part of the smaller locality to shift its burdens on to the
shoulders of the whole county.
 When it comes to highway construction, the high technical skill
which needs to go into the work is a commodity which comes too
high for a town and even, as we have pointed out, for the county.
But the most serious misfits of town government are the local
agents of tax administration. Wherever the town is the smallest tax
unit not only is the number of officials needlessly multiplied, but
diverse standards of property valuation are set up and competition is
resorted to between towns with a view to escaping their just share
of taxation. Without a dissenting voice the recognized tax experts of
the county are firmly of the opinion that the town as a unit of tax
assessment and tax collection must give way to the county.
And so, the readjustments that are working out the ultimate
destiny of the county are not wholly of a negative sort. It is not all a
matter of trimming the county’s wings.
[16] Variously designated in different states as state’s attorney,
prosecutor of the pleas, county attorney and solicitor.
[17] In Pennsylvania known as the “prothonotary.”
[18] This discussion of highway matters is based principally
upon a monograph by J. E. Pennybacker, Chief of Road
Economics, Office of Public Roads, Department of Agriculture. Y.
B. Separate, 1914.
 CHAPTER XV
COUNTY HOME RULE
Some counties indeed are awakening to a sense of their identity
and are asserting with much vigor their ability to organize and
manage many concerns which have been conspicuously mishandled
either by the state authorities or by the smaller local units.
Nowhere has the need for “readjustment” to meet this demand
been more keenly appreciated than in California. Elsewhere in these
chapters we have referred to the great diversity in the underlying
social and physical conditions in that state. To meet this situation
fifty-six “general” laws (that were not general at all) had been
enacted, for as many counties. No other method of individualizing
county government had been resorted to until 1911. At that time the
progressive leaders in the legislature wished to bring government, all
down the line, into sympathy with standards of simplicity and
efficiency that were then beginning to be accepted. For a time
county government seemed to present an insuperable obstacle: how
could a state system be devised that would square with these new
ideas? Then it was remembered that for upwards of thirty years the
cities of California had been determining for themselves what
municipal officers should be chosen, how they should be chosen and
what powers they should exercise. California was a pioneer in
municipal home rule and the system had worked pretty much to
everybody’s satisfaction.
Inasmuch, however, as counties have much more intimate
relations with the state government it seemed impracticable to allow
them quite so large discretion as the cities, in determining the
powers they should enjoy. And so, the California amendment gives
the people of the county freedom to determine the form and detail
of the county organization, subject to the proviso that each of the
necessary county officers such as sheriff, district attorney, etc., must
be maintained to execute the state law within the county. Members
of the county board of supervisors must be elected, but not
necessarily by districts. All other county officers, except the superior
court judges, may be either elective or appointive in a manner set
forth in a county charter.
The procedure by which California counties may take advantage of
the home-rule privilege is as follows: A board of fifteen freeholders is
elected, either in pursuance of an ordinance adopted by three fifths
of the members of the board of supervisors, or of a petition signed
by fifteen per centum of the qualified electors of the county,
computed upon the total number of votes cast therein for all
candidates for governor at the last preceding gubernatorial election.
Within one hundred and twenty days from the time their election is
declared the board of freeholders must prepare and cause to be
published a charter for the government of the county. Within sixty
days after its first publication (unless a general election intervenes)
the charter is submitted to the voters of the county for adoption or
rejection. It is then submitted to the legislature at its next session
for approval or rejection but not for amendment. But since a
California legislature in thirty-seven years has never been known to
reject a charter or a charter amendment of a city, the outlook for a
policy of county non-interference would seem to be good.
It may be, however, that the California plan is too radical a change
for states which have not yet granted freedom to their cities. A less
sweeping way of affording relief from iron-clad forms of government
is found in the statutes of Illinois, New Jersey and other states,
through which it is possible for any county to pass from one
prescribed form to another by petition and popular election. Similar
laws are in operation in a number of states permitting cities to adopt
the commission plan, and in four states the cities may make a choice
between three or four forms under an optional law.
 Following the passage of such a city law in New York, the County
Government Association and the official commissions on the
reorganization of government in Nassau and Westchester counties
memorialized the constitutional convention of 1915 for amendments
which would authorize the counties to adopt a plan of organization
suited to their local needs. These associations formulated the
question of county adjustment for “up-state” New York counties in
these words:

First: That the Legislature should be required by the

Constitution to provide optional plans of county government,
any one of which any county may adopt by a vote of the people.
Second: That the Legislature should in such plans confer
upon the Board of Supervisors or other governing body in such
county such powers of local legislation as the Legislature may
deem expedient.
Third: That the Constitution should require that no such plan
of government should be imposed on any county until approved
by the electors thereof and that no amendment to any plan of
government should affect any county which has previously
adopted such plan, unless such amendment is accepted by such
county, or unless such amendment relates to some state
function.
Fourth: That the Constitution should require that all laws
relating to the government of counties should be general both in
terms and in effect, except that special or local laws relating to
such government may be passed, but shall take effect only on
approval of the county affected.

The California amendment has been put to use in the four

counties of Los Angeles, San Bernardino, Tehama, and Butte, all of
which have their special “home-rule” charters. Its early use is
contemplated in Alameda, Napa and Santa Barbara counties. Since
1911 the scope of the amendment has been broadened so as to
permit of considerable latitude in the consolidation of city and county
governments.
Such is the counter-movement to centralized state control. In no
wise are the two in the least inconsistent, for the latter tendency is
to limit the subjects in which the county acts in the capacity of a
local state agent while the former concerns itself simply with
methods of performing service under local popular control.
The home-rule movement, if it may so be indicated, is practical
evidence that people are regarding counties as something more than
mere geographical expressions. Counties are thinking units. They are
capable of framing local policies. Therefore they would extend their
opportunities to think and to express themselves.
This idea seems to be at the bottom of the local option policy of
the organized anti-liquor forces throughout a great portion of the
country. Shrewd tactics, of course, has a good deal to do with it, for
county and other forms of local option are but the thin side of a
wedge to state-wide and even nation-wide prohibition. But for the
present at least, organizations like the Anti-Saloon League realize
that counties are very handy and convenient units of public
sentiment and have been instrumental in securing county option
laws in many states, including Idaho, Alabama, Kentucky, Louisiana,
Michigan, Minnesota, Montana, Oregon, Texas. Incidentally, the
county option plan makes the public policy district in liquor matters
coincide with districts for enforcement, thus minimizing the danger
of nullification.
 CHAPTER XVI
CONSOLIDATION
The battle cry of local freedom comes up loudest from urban
centers. The simple reason is that the counties were devised
originally for communities in a state of nature—a few people, widely
scattered, all but oblivious to the existence or need of government.
City communities on the other hand are highly complex,
individualized, differentiated units. Accordingly, their governmental
garments must be custom-made.
City governments indeed were instituted partly to escape the
strait-jacket inflexibility of the counties. Gradually, as we have seen,
they elbowed the county governments into a dark corner, to the
infinite debasement of the sheriff, the coroner, the poor master and
the tax collector and other typical accessories of the county.
But almost everywhere, at some point short of full county
annihilation, the pressure of the city stopped. Perhaps it was the
politicians who intervened, to save “the boys” at the court house; or
perhaps it was the feeling that seems to have settled down upon our
political thinking, that counties, like death and taxes, have to be.
Within very recent years, bold spirits in some of the metropolitan
centers have begun to feel that the county, in their particular
communities, was a public nuisance and have been “going” for it.
Thus the New York Times, when the New York Constitutional
Convention was in session in 1915, delivered not a few strokes for a
proposition to abolish existing boundaries of the sixty-one counties
and substitute therefor eight administrative districts. Cleveland,
Ohio, reformers would like to have that city divorce itself from the
rural part of Cuyahoga County. In Rochester, a recent survey has
suggested a similar course with reference to Monroe County. Studies
have also been made recently (1916) by the City Club of Milwaukee.
A member of the city commission in Jersey City has recently caused
to be passed in the legislature a bill providing for a vote on
consolidation of municipalities in Hudson County on a sort of
borough plan. In Cincinnati the question of consolidation of the city
with Hamilton County was recently opened, apparently for the first
time, in newspaper discussions. In the 1916 New York legislature
there was under discussion a bill extending control of the board of
estimate and apportionment in New York City over the employees of
the five counties within the city. This was in line with a recent report
by the chamberlain and the commissioner of accounts of New York
City submitted to the Constitutional Convention, which pointed out
the advantages of the abolition of counties in New York City and the
transfer of their functions to the control of the city authorities. St.
Louis City actually accomplished the fact in 1876 when it separated
from St. Louis County. Baltimore and a number of Virginia cities have
long been separated (for historical rather than reformatory reasons,
however,) from the surrounding rural or suburban territory.
In practice, the process of relocation of county boundary lines is
very much like the reversing of a long series of court decisions. Local
tradition and the gradual crystallizing of the interests of local
politicians militate powerfully to maintain the status quo. And, yet,
for all that, the shifting of boundary lines must inevitably come, if
local governments are to meet their obligations.
Just where and by what criteria the new lines are to be laid is no
easy question to decide; our metropolitan centers are of such
various origins and in such differing degrees of development. The
committee on City-County Consolidation of the National Municipal
League in a preliminary report rendered in 1916 seeks to classify the
urban communities in this fashion:

“The simplest type of urban county is that in which the

geographical limits of the two local units are identical and the
population has grown up out of a single well-defined historical
nucleus. Among other communities there would fall within this
 classification the cities of Philadelphia, Denver, San Francisco
and Baltimore and eighteen cities in Virginia.
“A second type is that in which a single city furnishes an
overwhelming proportion of the population, but occupies a
relatively small part of an otherwise rural territory. Among the
cities which fall in this classification are Buffalo, Milwaukee and
Cleveland.
“The third type is that in which a city of predominating size
and importance is surrounded by a number of smaller but
vigorous municipalities which have grown up not as suburbs of
the main city, but out of independent historical beginnings.
Cases in point are the two largest counties of New Jersey,
Hudson and Essex.
“A fourth type is one which contains several strong
municipalities, no one of which has achieved a position of
undisputed leadership. Alameda County, Cal., which contains the
cities of Oakland, Alameda and Berkeley will serve to illustrate.
“The most advanced type of the city-county problem involves
the adjustment of the political to the physical and social unity of
a great urban area, regardless of established boundaries of
either cities or counties. The metropolitan districts of New York
and Massachusetts are the most conspicuous illustrations of this
problem.”

The key to reconstruction is the same in every case: simplification.

Eliminate duplication of civil divisions; substitute one government for
two or many (in the case of Chicago, twenty-two!). There is a good
deal of logic in a separate local government to serve as a state
agency. But everywhere the people have waived the right or
privilege of a logical government when they have illogically insisted
upon selecting their officers locally. When the district attorney, for
instance, is chosen by the electors of the county he may be legally
the state’s representative but he is practically a local officer in very
much the same sense as the mayor. Then why, for legal fiction’s
sake, distinguish between the city and the county?
One American city-county has not only seen the inefficiency and
hypocrisy of the dual system but has actually wiped out the last
semblance of distinction between the two divisions. The story is told
in legal form in Article XX of the Colorado constitution:

“The municipal corporation known as the city of Denver, and

all municipal corporations and that part of the quasi-municipal
corporation known as the County of Arapahoe, in the state of
Colorado, included into the exterior boundaries of the said city
of Denver, as the same shall be bounded when this amendment
takes effect, are hereby consolidated and are hereby declared to
be a single body politic and corporate, by the name of the city
and county of Denver.”

By this same amendment the city and county were declared to be

a single judicial district of the state, and county officers were
disposed of by prescribing that

“the then mayor, auditor, engineer, council (which shall

perform the duties of a board of county commissioners), police
magistrate, chief of police and boards, of the city of Denver
shall become respectively, said officers of the city and county of
Denver, and said engineer shall be ex-officio surveyor and said
chief of police shall be ex-officio sheriff of the city and county of
Denver; and the then clerk and ex-officio recorder, treasurer,
assessor and coroner of the county of Arapahoe and the justices
of the peace and constables holding office within the city of
Denver, shall become, respectively, said officers of the city and
county of Denver, and the district attorney shall be ex-officio
attorney of the city and county of Denver.”

In 1913 an amendment to the Denver charter, adopted by popular

vote, provided for the commission form of government with the
usual divisions of administration into five departments respectively of
Property, Finance, Safety, Improvements and Social Welfare. To
some one of these commissionerships was assigned jurisdiction over
each of the several county officers, as appropriately as the
conditions would permit.
The process of amalgamation was made complete when in May,
1916, Denver abandoned the commission plan and made the mayor
chief executive of the county as well as of the city, with appropriate
appointing power.
In New York, consolidation has extended to most of the fiscal
functions, tending to leave intact only so much of the original county
structure as is incidental to the administration of justice, including
the courts themselves, the sheriff in his capacity of court executive
and the court clerks. The functions of the county treasurer have
been transferred to the city chamberlain and the comptroller, and
the department of taxes and assessments has been attached to the
city organization. The separate governing bodies of the five counties
have been swept away and their powers transferred to the city
Board of Estimate and Apportionment and Board of Aldermen. New
York City also has not only taken over the function of public
charities, but its Department of Corrections has been steadily
encroaching upon the prerogatives of the sheriffs. The future issue
of city consolidation there is accordingly reduced to a matter of
abolishing the five counties and transferring their functions to
officers under city control and making the independent elective
officers such as the sheriff, district attorney, clerk and register
appointive by either city or state authorities.
[19]In spite of much that remains to be done, consolidation has
proceeded to an advanced stage also in Boston, though hardly in a
direct line towards greater simplicity. Before it became a city in 1820
a conflict had arisen as to the jurisdiction of the town, and of the
court of sessions for Suffolk County, respecting highways and
taxation. The legislature thereupon abolished the court and
transferred its administrative functions to the mayor and council of
Boston. Boston, however, is not geographically identical with Suffolk
County, since the latter for the purposes of the administration of
justice, includes the city of Chelsea and the towns of Revere and
Winthrop. Boston has the title to all the real and personal property
of Suffolk County but also pays the entire expense of its
administration. The treasurer and auditor of accounts of the city act
in similar capacities for the county of Suffolk. But there are seven
elective county officials and several, virtually independent of each
other, who are chosen by the governor or the justice of the Superior
Court.
This situation as to the actual political Boston is confusing enough.
Metropolitan Boston on the other hand, comprises thirty-nine
municipalities situated in five counties, which make up a compact
community of a million five hundred thousand inhabitants. These
centers have every facility for communication and transportation.
And the state has indeed recognized the unity of the district by
establishing such agencies of administration as the metropolitan
park commission and the metropolitan water and sewer board.
San Francisco has proceeded so far as to have a single governing
body and a single set of fiscal officers for city and county.
With this, the recital of actual accomplishments toward formal
consolidation is about complete. The advantages of consolidation as
they appear to a disinterested observer are obvious. But the process
is usually difficult in the extreme, especially as it relates to the
equitable distribution of assets and liabilities of the parts to be
consolidated. New York City only accomplished this by assuming the
debts of the outlying counties and municipalities. But in the course
of nineteen years not even that generous concession has sufficed to
the vigorous local spirit of Brooklyn, the Bronx, Queens and
Richmond.

FEDERATION
Where immediate consolidation at a single stroke is out of the
question, as is apparently almost always the case, a more easy
transition is suggested by the recommendations of the City and
County Government Association of Alameda County, California.
Realizing that a powerful local sentiment in the outlying territory
militated irresistibly against annexation to the city of Oakland or, in
fact, any form of complete organic union of the municipalities in the
county, this Association has proposed as the logical first step to a
more economically organized county, a plan of federation.

ORGANIZATION CHART
For the City and County of Alameda and its Boroughs as
Proposed in the Tentative Charter Submitted by the City and
County Government Association
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookluna.com