Info. System & Analytics Sem-1 Module- 4
Info. System & Analytics Sem-1 Module- 4
Data mining is the process of analyzing hidden patterns of data according to different
perspectives for categorization into useful information, which is collected and
assembled in common areas, such as data warehouses, for efficient analysis, data
mining algorithms, facilitating business decision making and other information
requirements to ultimately cut costs and increase revenue.
Data mining is widely used in diverse areas. There are a number of commercial data
mining systems available today and yet there are many challenges in this field. In
this tutorial, we will discuss the applications and the trend of data mining.
The financial data in banking and financial industry is generally reliable and of high
quality which facilitates systematic data analysis and data mining. Some of the
typical cases are as follows −
Retail Industry
Data Mining has its great application in Retail Industry because it collects large
amount of data from on sales, customer purchasing history, goods transportation,
consumption and services. It is natural that the quantity of data collected will
continue to expand rapidly because of the increasing ease, availability and popularity
of the web.
Telecommunication Industry
In recent times, we have seen a tremendous growth in the field of biology such as
genomics, proteomics, functional Genomics and biomedical research. Biological data
mining is a very important part of Bioinformatics. Following are the aspects in which
data mining contributes for biological data analysis −
The applications discussed above tend to handle relatively small and homogeneous
data sets for which the statistical techniques are appropriate. Huge amount of data
have been collected from scientific domains such as geosciences, astronomy, etc. A
large amount of data sets is being generated because of the fast numerical
simulations in various fields such as climate and ecosystem modelling, chemical
engineering, fluid dynamics, etc. Following are the applications of data mining in the
field of Scientific Applications −
Intrusion Detection
Intrusion refers to any kind of action that threatens integrity, confidentiality, or the
availability of network resources. In this world of connectivity, security has become
the major issue. With increased usage of internet and availability of the tools and
tricks for intruding and attacking network prompted intrusion detection to become a
critical component of network administration. Here is the list of areas in which data
mining technology may be applied for intrusion detection −
1. Business understanding
First, it is required to understand business objectives clearly and find out what
are the business‘s needs.
Next, we have to assess the current situation by finding the resources,
assumptions, constraints and other important factors which should be
considered.
Then, from the business objectives and current situations, we need to create
data mining goals to achieve the business objectives within the current
situation.
2. Data understanding
First, the data understanding phase starts with initial data collection, which we
collect from available data sources, to help us get familiar with the data.
Some important activities must be performed including data load and data
integration in order to make the data collection successfully.
Next, the ―gross‖ or ―surface‖ properties of acquired data need to be examined
carefully and reported.
Then, the data needs to be explored by tackling the data mining questions,
which can be addressed using querying, reporting, and visualization.
Finally, the data quality must be examined by answering some important
questions such as ―Is the acquired data complete?‖, ―Is there any missing
values in the acquired data?‖
3. Data preparation
The data preparation typically consumes about 90% of the time of the project. The
outcome of the data preparation phase is the final data set. Once available data
sources are identified, they need to be selected, cleaned, constructed and formatted
into the desired form. The data exploration task at a greater depth may be carried
during this phase to notice the patterns based on business understanding.
4. Modelling
First, modelling techniques have to be selected to be used for the prepared
dataset.
Next, the test scenario must be generated to validate the quality and validity
of the model.
Then, one or more models are created by running the modeling tool on the
prepared dataset.
In the evaluation phase, the model results must be evaluated in the context of
business objectives in the first phase. In this phase, new business requirements may
be raised due to the new patterns that have been discovered in the model results or
from other factors. Gaining business understanding is an iterative process in data
mining. The go or no-go decision must be made in this step to move to the
deployment phase.
6. Deployment
The knowledge or information, which we gain through data mining process, needs to
be presented in such a way that stakeholders can use it when they want it. Based on
the business requirements, the deployment phase could be as simple as creating a
report or as complex as a repeatable data mining process across the organization. In
the deployment phase, the plans for deployment, maintenance, and monitoring have
to be created for implementation and also future supports. From the project point of
view, the final report of the project needs to summary the project experiences and
reviews the project to see what need to improved created learned lessons.
Text Analytics is the process of converting unstructured text data into meaningful
data for analysis, to measure customer opinions, product reviews, feedback, to
provide search facility, sentimental analysis and entity modelling to support fact
based decision making. Text analysis uses many linguistic, statistical, and machine
learning techniques. Text Analytics involves information retrieval from unstructured
data and the process of structuring the input text to derive patters and trends and
evaluating and interpreting the output data. It also involves lexical analysis,
categorization, clustering, pattern recognition, tagging, annotation, information
extraction, link and association analysis, visualization, and predictive analytics. Text
Analytics determines key words, topics, category, semantics, tags from the millions
of text data available in an organization in different files and formats. The term Text
Analytics is roughly synonymous with text mining.
Text analytics software solutions provide tools, servers, analytic algorithm based
applications, data mining and extraction tools for converting unstructured data in to
meaningful data for analysis. The outputs, which are extracted entities, facts,
relationships are generally stored in a relational, XML, and other data warehousing
applications for analysis by other tools such as business intelligence tools or big data
analytics or predictive analytics tools.
1. Text mining, Text parsing, Text Identification, Text extraction, Text categorization,
Text clustering.
2. Extraction of concepts, entities, relations, events.
3. Creation of taxonomies.
4. Search Access, Web crawling, and indexing, duplicate document identification.
5. Analyze all major file formats and all major languages- Natural
Language/Semantic Toolkits.
6. Entity relation modelling.
7. Link analysis, link text repositories.
8. Ability to identify and analyze sentiments, people, places and other information
from websites, internal files, reports, surveys, forms, employee surveys, claims,
underwriting notes, medical records, emails, news, blogs, social media, customer
surveys, market surveys, online forums, online reviews, review sites, scientific
journals, website feedback, call centre logs, transcripts, snail mail, sales notes.
9. Document summarization features and records management.
10. Interactive visualization.
1. Sentiment Analysis
2. Search access of unstructured data
3. Email spam filters to determine the characteristics of messages to filter that are
likely to be advertisements or promotional, phishing or unwanted material
Text Mining
Text mining may be defined as the process of analyzing data to capture key
concepts and themes and uncover hidden relationships and trends without prior
knowledge of the precise words or terms that authors have used to express those
concepts.
Text Mining is the process of examining data to gather valuable information. Text
mining, also known as text data mining involves algorithms of data mining, machine
learning, statistics, and natural language processing, attempts to extract high quality,
useful information from unstructured formats. Text mining, which is often
interchangeably used with ―text analytics‖ is a means by which unstructured or
qualitative data is processed for machine use.
Text mining techniques are rapidly penetrating the industry, right from academia and
healthcare to businesses and social media platforms. Here are a few applications of
text mining being used across the globe today:
1. Risk Management
One of the primary causes of failure in the business sector is the lack of proper or
insufficient risk analysis. Adopting and integrating risk management software
powered by text mining technologies such as SAS Text Miner can help businesses
to stay updated with all the current trends in the business market and boost their
abilities to mitigate potential risks. Since text mining technologies can gather relevant
information from across thousands of text data sources and create links between the
extracted insights, it allows companies to access the right information at the right
moment, thereby enhancing the entire risk management process.
Text mining techniques, particularly NLP, are finding increasing importance in the
field of customer care. Companies are investing in text analytics software to enhance
their overall customer experience by accessing the textual data from varied sources
such as surveys, customer feedback, and customer calls, etc. Text analysis aims to
reduce the response time of the company and help address the grievances of the
customers speedily and efficiently.
4. Business Intelligence
Organisations and business firms have started to leverage text mining techniques as
a part of their business intelligence. Apart from providing profound insights into
customer behaviour and trends, text mining techniques also help companies to
analyse the strengths and weaknesses of their rivals, thus, giving them a competitive
advantage in the market. Text mining tools such as Cogito Intelligence
Platform and IBM text analytics provide insights on the performance of marketing
strategies, latest customer and market trends, and so on.
There are many text mining software packages designed exclusively for analysing
the performance of social media platforms. These help to track and interpret the
texts generated online from the news, blogs, emails, etc. Furthermore, text mining
tools can efficiently analyse the number of posts, likes, and followers of your brand
on social media, thereby allowing you to understand the reaction of people who are
interacting with your brand and online content. The analysis will enable you to
understand ‗what‘s hot and what‘s not‘ for your target audience.
Web Mining
Web mining is the process of using data mining techniques and algorithms to extract
information directly from the Web by extracting it from Web documents and services,
Web content, hyperlinks and server logs. The goal of Web mining is to look for
patterns in Web data by collecting and analyzing information in order to gain insight
into trends, the industry and users in general.
Web mining is a branch of data mining concentrating on the World Wide Web as the
primary data source, including all of its components from Web content, server logs to
everything in between. The contents of data mined from the Web may be a collection
of facts that Web pages are meant to contain, and these may consist of text,
structured data such as lists and tables, and even images, video and audio.
1. Web content mining — This is the process of mining useful information from
the contents of Web pages and Web documents, which are mostly text,
images and audio/video files. Techniques used in this discipline have been
heavily drawn from natural language processing (NLP) and information
retrieval.
Web mining also enable web based establishments to give better access to other
services or adverts. When an establishment makes an advertisement for services, it
does provide, but are provided by different companies. Usage of mining data will
give most effective to those paths to those portals. There are three uses for mining
in this fashion.
The first is processing used to complete discovery. This is also the most difficult use
because only bits of information like user information, IP addresses and site sites are
available. With this small amount of information that is available, it gets harder to
trash the user through a site; this program can be able to follow users throughout
pages of sites. Content use is content processing, this consist of some conversations
or web information like text, images, scripts and other useful forms. With the
clustering and categorization of web page helps in page information bases on titles,
specified contents and images available.
The information that is gathered may combine registration of the user, access
logs, and information that can lead to better web site structure. Therefore, providing
the company to the most valuable in online marketing, these can give some
benefits of external marketing of a company and its products, services and overall
management.
Social media analytics is the process of collecting and analysing audience data
shared on social networks to improve an organization's strategic business decisions.
Social media analytics help marketers with a number of tasks, from informing their
strategy to planning campaigns and inspiring content ideas. There are five major
benefits of tracking social analytics:
1. Trend spotting
Trend spotting is the act of pinpointing upcoming trends before they're mainstream.
Keeping a close eye on your social media analytics can help you do just that. Some
of the trends that your social media analytics can help you determine include:
If analysed properly, your social media analytics reports can be a huge help in
identifying what you should post more of, what types of content are becoming more
popular and what your audience wants to hear more about in the next quarter or
year.
2. Brand sentiment
Brand sentiment illustrates how people are feeling about your brand. It includes all
positive, neutral and negative feelings that are discussed online. By looking through
your social media analytics, you can review and measure your brand sentiment
through a sentiment analysis software.
This helps ensure your audience is happy with your business and enables you to
detect opportunities to make amends with unsatisfied customers. And you can
uncover opportunities to improve your business.
For instance, through sentiment analysis you could discover your customers are
asking the same questions about a particular product feature, enabling you update
your FAQ page or help centre. Sentiment analysis can be used with competitor
analysis because you can pinpoint new competitors and related topics your
customers are buzzing about that you may have not considered before.
Value perception (or perceived value) refers to the overall customer opinion of your
brand's product or service and whether or not it can meet their needs. Perceived
value is key to determining demand and the price point of a product or service. For
example, if your product has a low perceived value, customers won't be willing to pay
much for it.
You can measure value perception by using social listening tools and monitoring
data from other digital marketing dashboards, such as Google Analytics. This can
help guide the content you create to improve value perception and make sure you're
showcasing how your product or service can hit key pain points.
Social media analytics can also help you see which channels and content are
performing well, so you can create actionable, realistic social media goals and
objectives.
The key word here is realistic. If you take a look at your social media analytics
reports and realize your Instagram account is growing by 10 followers per week,
trying to jump from 5,000 followers to 10,000 followers in a single quarter is not a
realistic goal, even if you revamp your posting strategy. You might instead try to
make a goal where your account starts growing by 20 followers per week instead
and steadily increase that goal from there.
5. Proving ROI
Finally, your social media analytics can help prove the ROI of your social media
marketing efforts. Each time you run a new campaign, monitor your social analytics
to see how the content is performing, if people are clicking over to your website and
if you're generating new sales. Doing this demonstrates social media ROI so teams
can earn more buy-in and resources. UTM tracking and URL shortening are two
ways that make proving ROI via analytics even easier. This way, you can attribute
specific pipeline and purchases to your social media efforts.
There are several different types of social media analytics you should monitor in
your social media dashboard that will guide your strategy and discover valuable
insights. We'll walk you through the six main types of analytics below.
1. Performance analysis
First and foremost, you need to measure the overall performance of your social
media efforts. This includes social media metrics including:
Impressions
Reach
You can easily gauge all of this within your Sprout Social dashboard:
You can use Sprout‘s reports to showcase how your marketing content is performing
and if you need to switch up the content you're posting. You can also identify which
types of posts resonate with your audience best.
2. Audience analytics
Next, you'll want to take a look at your audience analytics. This will help you discover
which demographics your content is reaching—and ensure they match up to your
target audience. If not, you may need to adjust your content strategy to better attract
your ideal customer profile.
Age
Gender
Location
Device
With Sprout Social, you can also gather audience analytics about specific topics
related to your industry, which can help you build out your customer profiles. Here's
an example of what that could look like for one of your topics:
3. Competitor analysis
Another key area to look into is how your competitors perform on social media. How
many followers do they have? What is their engagement rate? How many people
seem to engage with each of their posts?
You can then compare this data to your own to see how you stack up—as well as set
more realistic growth goals. Using a tool like Sprout, you can gather all of this data in
one place and measure it network by network.
Pay attention to how your benchmarks stand up to your competitors and consider
adjusting your social media strategy to take advantage of opportunity gaps.
When you're putting money behind specific social media posts, you want to make
sure they're driving results. This is why you absolutely need to pay close attention to
your paid social analytics.
Each social media platform that you run ads through will have its own dashboard to
provide you with all of this information, but you may want to create your own spread
sheet as well to track total ads and ad spend.
5. Influencer analysis
This can help you gauge overall engagement from your influencer campaigns. If you
have an affiliate marketing program, you can designate promo codes for each
individual influencer to use so your team can track how many sales each partner
drives as well.
6. Sentiment analysis
The last major segment of social media analysis you'll want to track is brand
sentiment. Earlier, we talked about how social media analytics tools can help you
determine and measure sentiment analysis. But if you want to dig even deeper,
use social listening to gauge specific connotations around your brand.
There are a heap of social media analytics tools to choose from, but it‘s all about
finding the platform that fits the unique needs of your organization. Take a look at our
top three suggestions:
1. Sprout Social
Sprout offers cross-channel social media analytics, enabling you to dig into your
performance on a single network or compare results across various networks at
once.
But that‘s just the tip of the iceberg for our social media analytics. With Sprout‘s
Advanced Listening tool, you can conduct sentiment analysis and uncover data
about your audience, share of voice and relevant topics. And with Sprout Social
Influencer Marketing (formerly Tagger), you can measure and maximize your
influencer marketing ROI and optimize your social marketing efforts.
2. Google Analytics
Google Analytics isn‘t solely for social analytics, but it‘s a staple for social media
practitioners and leaders. You can create reports to monitor:
How your social media efforts drive web traffic and lead generation
Which social networks fuel the most traffic
Audience demographics
ROI of your social media campaigns
3. Rival IQ
Rival IQ is another tool for customized social media analytics reports. The platform
can help brands track their success across a number of social networks including
YouTube. Rival IQ also provides competitor analysis, social listening, influencer
tracking, hash tag analytics and social media audits. Rival IQ is a great option for
businesses with multiple social media channels or agencies working with multiple
clients.
Sentiment Analysis
Sentiment analysis is a type of data mining that measures the inclination of people‘s
opinions through natural language processing (NLP), computational linguistics and
text analysis, which are used to extract and analyse subjective information from the
Web - mostly social media and similar sources. The analysed data quantifies the
general public's sentiments or reactions toward certain products, people or ideas and
reveal the contextual polarity of the information.
Sentiment analysis uses data mining processes and techniques to extract and
capture data for analysis in order to discern the subjective opinion of a document or
collection of documents, like blog posts, reviews, news articles and social media
feeds like tweets and status updates.
1. Reputation management - or you could also call it brand monitoring. We all know
how much good reputation means these days when the majority of us check social
media reviews as well as review sites before making a purchase decision. I‘ll be
honest with you - I don‘t remember when was the last time that I decided to eat out
without checking the reviews of a place beforehand. The same thing applies to
buying stuff online or researching tools I use daily at work as a marketer.
Negative reviews put people off and how you handle can define your future as a
business. You could either ignore them (highly not recommended), act rude and
make your situation even worse, or apologise for whatever caused a person to write
a negative opinion and do your best to make up for it.
But you have to be aware of those opinions in the first place. That‘s where social
media monitoring combined with sentiment analysis comes in! While some say it‘s
just a fad or something that only big businesses can use, I believe a social media
monitoring tool not only will help you manage your reputation, but also prevent your
customers from turning to your competitors and earn you money they could spend
elsewhere.
Such mentions will appear in your dashboard with a flashing red colour, and you
better start engaging them as soon as they are there.
3. Competitor monitoring - chances are some of your competitors are getting bad
press online. It‘s where you could step in as long as you‘re aware of those negative
mentions. I don‘t advise to take advantage of whatever they had neglected in an
aggressive way, but I highly recommend chiming in conversations when they don‘t
even bother to reply to the mentions they are getting.
Marketing has always been an expensive affair for businesses. They spend a
significant percentage of their budget on marketing. Using business analytics
aspects will ensure that businesses get the maximum ROI on their spending.
A data-driven marketing strategy will help these businesses optimize their marketing
campaigns to target the specific aspects of their customer‘s journey. With targeted
marketing, the results improve, as there is less ―shooting blind arrows‖ and more of
approaching the right customers.
2. Creating Surveys
With the help of business analytics, marketers can always ask the right questions
from their customers and target audience. The importance of this step is visible when
creating the solution. Because if you can understand what your customers need, you
are better able to create the desired solution.
Without access to data, you can keep on creating endless surveys and
questionnaires and then analyze the answers to identify the results. But with access
to data, you can gain insights prior to creating surveys.
Using business analytics helps accelerate the surveying process and set the right
KPIs.
Businesses must understand that not every person is their customer. There are a
limited number of people who need their product. It will be more productive if a
business entity will market its product in front of interested customers rather than
including everyone in the mix.
With the help of business analytics tools and techniques, you will find it easier to
categorize your ideal customers. You can create a customer persona and build
brand messaging that is closely associated with these individuals.
You can go on to collect information from different channels and classify them with
business analytics tools. As a result, you can evaluate and create a pre-verified list
When you need to understand your customers, business analytics is one of the best
solutions. It will allow you to tailor your services and solutions according to your
customers‘ choices and preferences. From here, you can address the issues your
customers are facing and project your product as the panacea for all the issues.
You can drive better content that is relevant to your customers and also share helpful
content. All these aspects come together to form a cumulative marketing mix that will
bring effective results.
Based on your customer‘s buying behaviour and patterns, you can further create
strategies or modify them for better conversions, higher visibility, and enhanced
sales.
For the sales team, business analytics is equally important as it is for the marketing
team. The key component here is the insights and trends we can deduce by using
business analytics techniques. Some of the insights include creating bespoke pricing
strategies, knowing the customer‘s response, and enhancing public engagement.
Here‘s how we can use business analytics in sales.
The right pricing strategies will set your product apart from the competition and
attract higher sales. However, to set the right price and provide the best value to
your customers, it is important to take the help of business analytics for setting the
right price.
Using business analytics will help a business eliminate the ―sounds about right‖
pricing strategy. This will help them move towards a value-based pricing system.
Another way to use business analytics in the sales process is identifying the strong
and weak products. In this, the business can determine the trending products and
the ones that are seeing a declining trend in terms of sales.
With this, the sales team can find out which products are lagging behind due to
competition or which ones do not fit with the current market margins. Having this
information in hand means businesses can identify how long it takes for a deal to
close. It can show the results of specific sales campaigns and which elements in a
product can spark the customer‘s interest.
Business analytics has the potential to transform the sales process and pipeline. It
will provide a historical and predictive data set that businesses can use to identify
better opportunities and prospects.
Conclusion
Business analytics plays a key role in helping businesses and companies redefine
their sales, revenue, marketing, product creation, and operations. It has the power to
determine suitable opportunities for the business and leverage the data
understanding to deliver the target audience‘s specific needs.
Analysing and understanding data is always beneficial for a business. It will help
them streamline a lot of business-related processes and operations. However, to get
the best results, businesses need the services of a Business analyst, who can help
them interpret the data and identify trends and insights.
HR Analytics
The data software also collects and analyses information about employee-client
relationships, time management and work output. Once you figure out what drives
the best employees to perform at a higher level, you can apply this knowledge to
help other professionals in the company achieve similar results.
It‘s no secret that your job as an HR executive contains a ton of paperwork. Data
analytics uses many storage software systems, such as SQL to build databases.
You are able to collect, store, scrutinise, modify and analyse data with a few clicks of
your fingertips. It takes away some of the pain of filing large amounts of employee
and company information, which can be a tedious task.
During the appraisal process, you need an accurate analysis of every employee‘s
achievements, work habits, office behaviour and skill development. With business
analytics, you will already have digital methods that have tracked the employee‘s
data throughout the year. This makes your job during the annual employee review
easier.
You have data-backed documentation of the employee, which ensures the process
is authentic and transparent. By examining the information, identifying patterns and
comparing the results, you are in a better position to determine promotions and
increments. You also have the facts and figures to provide constructive feedback for
areas that need improvement.
Financial Analytics
Financial analysts work in various segments and across almost all industries. Most
businesses employ some sort of financial analyst, but other types of companies such
as investment banks, mutual funds, hedge funds, insurance companies, and other
financial institutions may employ a great deal more of them.
A financial analyst‘s primary role is to analyse financial data and trends to support
the decision-making processes. The specific responsibilities of a financial analyst
can vary depending on their role and where they work but generally include:
Key Takeaways
Manufacturers help keep their plant equipment running during production runs
by analyzing sensor data to recognize when such equipment will likely fail.
Manufacturers considering a transition to more service-oriented business
models use analytics to identify revenue streams that production inefficiencies
directly affect.
Analytics helps manufacturers continuously monitor their supply chains, giving
them visibility into the movement of raw materials or parts in transit from
suppliers as well as materials located at various plants.
Manufacturers use analytics to reduce the number and scope of product
recalls by identifying specific machines or production lines where quality
issues occurred. This lets manufacturers recall only specific product batches
rather than entire shipments.
Manufacturers use analytics to stay on track of crucial key performance
indicators to ensure they reach their perfect order targets.
To benchmark progress, you set KPI targets. Organizations typically base their
targets on one or more of the following:
Competitor performance
Past performance
Pre-determined benchmarks
KPIs provide targets to aim for, milestones to gauge progress, and insights to help
guide data-driven decision-making. By monitoring KPIs, organizations can identify
areas of strength and weakness and take actions to optimize performance.
Financial KPIs are among the most widely utilized measures. They are measures
that companies use to monitor, evaluate, and assess their financial health. This is
great for benchmarking, and it can show you how well your company is doing
compared to its goals, past performance, and competition.
Profitability:
Net Profit margin:
Return on Assets:
Return on Equity:
Liquidity:
Networking capital:
Current Ratio:
Solvency:
Efficiency:
These key performance indicators focus on tracking the company‘s relationship with
customers and its effects.
The main focus of these key performance indicators is to determine whether or not
the culture in the workplace is positive and conducive to production as well as note
areas for improvement with regards to workforce management.
Training hours per employee: The number of hour‘s employees has been
trained for their relevant skills.
Internal Promotions vs External Hires: This represents the company‘s ability
to train and nurture skilled employees for better roles.
It is calculated by dividing the number of internal promotions for new positions
that arose by the number of external hires.
Range Penetration: Range penetration is a pay metric that compares an
employee‘s salary to the overall pay range for their job or similar jobs at other
organizations. This pay comparison shows how much the employee‘s
compensation has progressed within the pay range.