Data mining - automation or not

Automated data mining gives marketing managers a tool to perform analyses that otherwise would
need to be handled by a highly trained researcher. This is accomplished by establishing a predetermined
analysis methodology. An algorithm is developed that attempts to mirror the step-by- step decision-
making process that a trained modeler would follow. At each step in the process, preset criteria are used
to select analysis options. Because experts have programmed these criteria, the results should be on par
with an expert analysis.

However, in data mining and modeling there are many parts of the process that cannot be automated,
including choosing a methodology to match a business problem, selecting a data set, quality checking
and preparing the data for analysis, choosing among the available options within the analysis process,
and interpreting and presenting the results.

An analysis could be automated when:

 The data being used is from a familiar source,

Within these guidelines, automation can provide huge advantages in time and cost. One clear example is
real-time data mining and decision support for Web marketing applications. Rather than establishing
fixed business rules, data can be analyzed in real time to evaluate and optimize custom content delivery
systems.

The applications do not have to be this high tech to be valuable. Direct marketing campaigns may need
to be checked repeatedly for performance against a number of targeting dimensions, such as
demographics and model scores. Automating a model that identifies high-performing segments would
allow the marketer to quickly evaluate and modify marketing strategies. It also allows the technical
analyst to focus on more complicated and challenging projects. The advantages of this automation are
even greater if the data mining software is integrated with the campaign management and tracking tool.
Necessity of automated data mining

The main reason for the necessity of automated computer systems for intelligent data analysis is the
enormous volume of existing and newly appearing data that requires processing. The amount of data
accumulated each day by various businesses, scientific, and governmental organizations around the
world is appalling. According to GTE research, scientific organizations store about 1 terabyte of new
information each day. It is impossible for human analysts to cope with such overwhelming amounts of
data.

Two problems that surface when human analysts process data are the inadequacy of the human brain
when searching for complex dependencies in data, and the lack of objectiveness in their analysis.
Therefore, one of the benefits of using automated data mining systems is that this process has a much
lower cost than hiring an army of highly trained and paid professional statisticians. Although data mining
does not completely eradicate the need for humans, it allows an analyst who has no programming and
statistics skill to extract knowledge from databases.
Web analytics and Business research

Web analytics is the process of gathering and analyzing your web content’s data in order to gather
meaningful information about how your site is being utilized by your users. There are plenty of Web
analytics applications out there, and you probably already know the leading players such as Google
Analytics, Crazy Egg, and remote-site services such as Alexa and Compete.

Web analytics is not just a tool for measuring Web site traffic. Web analytics applications can also help
companies measure the results of traditional print advertising campaigns. Web analytics helps you to
estimate how the traffic to the site changed after the launch of a new advertising campaign. Web
analytics provides data on the no of visitors, page views etc to gauge the popularity of the sites.

Further, web analytics gives information on competition. Comparison with competitors is a fundamental
element of business; even innovators need to know how far ahead they are in their market. The Internet
seems to offer fertile terrain for capturing accurate marketing statistics on website usage and position
relative to other players in a given market. Indeed, most of us have often heard web statistics from
Nielsen//NetRatings, Alexa or comScore cited in the press and elsewhere. Practitioners of Search Engine
Optimization and web marketing know that web analytics is not just silo analysis of a company’s
website: it also entails looking at how a website and its business performance metrics measure up in the
overall web ecosystem. This way web analytics helps in business research also.
Web Analytics is failing to drive business decision making

According to research published this week by E-consultancy and Lynchpin, Companies are typically
failing to adopt a coherent strategy which ties in web analytics data with business objectives,

Less than a fifth of companies (18%) have an internal strategy that ties data collection and analysis to
business objectives. According to the survey, more than half (56%) of responding organizations said that
they are "working on this," while a further 22% say that they don't have such a strategy.

The research also found that only a quarter of company respondents (25%) say that their web analytics
"definitely" provide actionable insights, with a further 56% saying that this is only sometimes the case.

Spending on technology accounts for 45% of company spending on web analytics compared to 18% for
consulting and services and a further 36% on internal staff.

Linus Gregoriadis, head of research at E-consultancy, said: "Respected analytics experts advise that the
amount spent on technology should only really be the tip of the iceberg when it comes to analytics
investment.

"Our research shows that many organisations are under-investing in internal analytics staff and failing to
implement a coherent measurement strategy which can help them turn their data into something of
real value to their business."

The majority of company respondents (58%) say that half or less of their web analytics data is useful for
driving decision-making. More than a third (38%) say that 30% or less of their data is useful.

The most widespread use of web analytics tools is for reporting traffic figures, the research found.
Almost nine out of ten responding organisations (88%) said that this is an "important use" of web
analytics.

The next most important function of analytics is to help understanding of customer behaviour, deemed
to be an important use by 70% of company respondents.

More than 700 companies took part in the survey including 434 "client-side" respondents and 229
supplier-side companies (including agencies, consultancies and analytics vendors).

Andrew Hood, managing director of Lynchpin added: "Analytics' reputation is in tatters. Millions have
been invested by large corporations around the world in analytics software that does not give business
the answers it promised. "Today, there is a massive disconnect between the analytics market and what
business needs. It is vital that the analytics sector address this issue and provide the business
consultancy and online strategy needed to unlock the potential of analytics software."
Web-based market research: Issues to Consider

The factors involved in selecting a web-based survey tool will vary greatly from research project to
research project. However, five general matters need to be considered when differentiating between
products.

 User Control: The ability for the researcher to manipulate the “look and feel” of survey items is
often an important consideration. For example, the ease of editing the contents of a survey item
or manipulating the layout of the survey is imperative. Many survey applications include
templates of standard survey items (e.g., select only one, select all that apply, text response
boxes) that the researcher uses to build the survey.

 Ease to Customize: In many situations, templates of standard survey items are suitable.
However, due to the nature of some research questions, standard survey items may not be
appropriate for collecting some types of data. Therefore, it might be important for the
researcher to use non-standard data collection items (e.g., rank order a series of items) that
cannot be created using templates.

 Turnaround Time: The timeframe between when a researcher receives initial approval to
conduct a study (e.g., IRB, funding agency), and the time that the investigator needs to begin
data collection is often an important consideration. This is especially true if the researcher needs
to conduct any type of pilot test during the early stages of the project.

 Analysis Tools: Many researchers prefer to extract data from the survey tool and conduct the
analysis using a standalone software application (e.g., SPSS). However, some web-based survey
tools have built in reports that allow the investigator to review summary results while the
survey is still active (i.e., no need to export data from the survey application and import it into
another product). Most of these internal reports provide basic demographics (e.g., number of
respondents, percentage responding “yes” to a certain item).

 Researcher’s Cost: The cost of using varying survey applications will be dependent upon the
arrangement the researcher negotiates with external vendors or organizational policies related
to cost recoupment for use of institutional resources. For many research projects, cost
considerations become a primary factor in selecting a survey tool.