Data and Information Management,

Journal 2017;
xyz 2017; 1 (2): AoP
122–135

The First Decade (1964-1972)

Panel Article Open Access
Research Article
Eric Zheng*, Yong Tan, Paulo Goes, Ramnath Chellappa, D.J. Wu, Michael Shaw, Olivia Sheng,
Max Musterman, Paul Placeholder
Alok Gupta
What Is So Different About
When Econometrics Meets Machine Learning
Neuroenhancement?
Was ist so anders am Neuroenhancement?
https://doi.org/10.1515/dim-2017-0012
received June 24, 2017; accepted August 20, 2017.
2 Topic 1: What is the meaning
Journal xyz 2017; 1 (2): 122–135
Pharmacological and Mental Self-transformation in Ethic
Comparison
of combining econometrics with
Keywords: econometrics, machine learning, information
The First
system, Decade
prediction, (1964-1972)
explanation, machine
integration. Pharmakologische learning?
und mentale Selbstveränderung im
ethischen Vergleich
Research Article
2.1 Eric Zheng
1 Max
Introduction
Musterman, Paul Placeholder
https://doi.org/10.1515/xyz-2017-0010
received February 9, 2013; accepted March 25, 2013; published online July 12, 2014
In the past few years, we have seen growing interests
What Is So Different About
Econometrics and machine learning used to perform Abstract: In theinconcept
well combiningof the aesthetic formation
econometrics of knowledge
with machine and its as soon
learning in

Neuroenhancement?
as possible and success-oriented application,
in their own orbits separately. They differ in purpose, research. In this panel, we would like to discuss what insights and profits without the
focus, and methodologies. However, due to the referenceabundant to the arguments developed around 1900. The main investigation also
we mean by saying “when economics meets machine
Was ist so anders am Neuroenhancement? includes the period between the entry into force and the presentation in its current
supply of “big data” and the demand for solving complex learning”. We will answer why this is important and
version. Their function as part of the literary portrayal and narrative technique.
problems, there emerges a trend of applying econometrics what we could do in a research. We are happy to have the
and machine learning in an integral manner. At Keywords:
the 11thFunction, transmission,
great panel investigation,
consisting of Pauloprincipal, period
Goes, Yong Tan, Ramnath
Pharmacological and Mental Self-transformation in Ethic
China Summer Workshop on Information Management Chellappa, D.J. Wu, and Michael Shaw.
Comparison
(CSWIM
Dedicated to Paul Placeholder
2017) in Nanjing, China, the meeting’s chairs My first question to our panelists is what we mean by
Pharmakologische
assembled a panel of seniorund mentale
information Selbstveränderung
system (IS) saying “combining im econometrics into machine learning”.
ethischen
scholars Vergleich
to discuss the theme “When Econometrics Meets What would qualify as a good paper that combines
1 Studies and Investigations
Machine Learning”. The panel consisted of Paulo Goes, econometrics with machine learning? I would like to invite
Yong Tan, Ramnath Chellappa, D.J. Wu, andTheMichael
https://doi.org/10.1515/xyz-2017-0010 Yong Tan to help answer the question first.
main investigation also includes the period between the entry into force and
Shaw. EricFebruary
received Zheng was theaccepted
9, 2013; panel coordinator. Olivia
March 25, 2013; Sheng
published online July 12, 2014
the presentation in its current version. Their function as part of the literary por-
and Alok Gupta also participated in the discussion trayalofand
thenarrative technique.
Abstract: In the concept of the aesthetic formation of knowledge and its as soon
panel. The panel discussed the guidelines and key issues 2.2 Yong Tan
as possible and success-oriented application, insights and profits without the
of combining econometrics and machine learning in IS
reference to the arguments developed around 1900. The main investigation also
research. This paper is a narrative of what the*Max Musterman:
panelists Institute
2.2.1 of Marine
Idea Biology,
1: Integration National Taiwan Ocean
of machine University,
learning 2 Pei-Ning
and
includes the period between the entry into force and the presentation in its current
Road Keelung 20224, Taiwan (R.O.C), e-mail: [email protected]
discussed in the panel discussion. Paul Placeholder:econometrics
of Marinewill be National
of great value
version. Their function as part of the literary portrayal andInstitute
narrative Biology,
technique. Taiwan Ocean University, 2 Pei-Ning
Road Keelung 20224, Taiwan (R.O.C), e-mail: [email protected]
Keywords: Function, transmission, investigation, principal, Actually,
period I have been talking about this topic many
Open Access. © 2017 Mustermann and Placeholder, published by De Gruyter.
times
This work is
licensed under the from
Creativethe methodological
Commons point of view. However,
Attribution-NonCommercial-NoDerivatives this time
4.0 License.
*Corresponding author: Eric Zheng: Jindal School of Management,
Dedicated to Paul Placeholder I would like to start with a disclaimer that methodology
University of Texas at Dallas, Richardson, TX, USA. E-mail:
[email protected] itself is just a tool. We want to advance the tool in order to
Yong Tan: Foster School of Business, University of Washington, solve more interesting business problems. The objective of
Seattle, WA, USA using econometrics is to find out the causal relationship
1 Goes:
Paulo Studies and
Eller College Investigations
of Management, University of Arizona, of data, which is our main purpose. On the other hand,
Tucson, AZ, USA
from the machine learning’s point of view, we want to
Ramnath Chellappa:
The main investigation also includes
Goizueta Business School,the period
Emory between the entry into force and
University,
Atlanta, GA, USA achieve predictive accuracy. Therefore, if we probe into
the presentation in its current version. Their function as part of the literary por-
D.J. Wu: Ernest Scheller Jr. College of Business, Georgia Institute of the two approaches deeper, it can be realized that there
trayal and narrative technique.
Technology, Atlanta, GA, USA are probably the same philosophical ideas that accurately
Michael Shaw: Gies College of Business, University of Illinois at describe the data generation process. Obviously, once
Urbana-Champaign, Champaign, IL, USA
we get to know the way in which data are generated, we
Olivia Sheng:
*Max David Eccles
Musterman: School
Institute of Business,
of Marine University
Biology, Nationalof Utah, Ocean University, 2 Pei-Ning
Taiwan
can get more ideas about the causal relationship in this
SaltRoad
LakeKeelung
City, UT,20224,
USA Taiwan (R.O.C), e-mail: [email protected]
Alok Gupta:
Paul Carlson School
Placeholder: of Management,
Institute University
of Marine Biology, of Taiwan Oceanprocess.
National University,On the other hand, if we can get to know how
2 Pei-Ning
Minnesota, Minneapolis, MN, USA
Road Keelung 20224, Taiwan (R.O.C), e-mail: [email protected] the data are generated and where it comes from, we can
2017; 1 (2): 122–135
Open Access.
Open Access.© Eric Zheng et
© 2017 Mustermann andal., publishedpublished
Placeholder, by De Gruyter Open.
by De Gruyter. Thiswork
This workis is licensed under the Creative Commons
licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives
Attribution-NonCommercial-NoDerivatives 3.0 License. 4.0 License. Unauthenticated
Download Date | 2/27/18 1:28 PM
2 Eric Zheng et al.

achieve higher predictive accuracy. Therefore, that is my In the past few years, we have had a typical paradigm
point of view about the integration of machine learning in machine learning. It is just like making a choice in a
and econometrics. black box, in which causal relationships and explanatory
features have been de-emphasized. Black boxes are
actually fine since we do not care about the underlying
2.2.2 Idea 2: Technical integration will be a substantial relationships inside. What we are interested in is
integration developing a model that can provide us a solution. All we
need is solution and prediction, and all the data are used
Now I would like to answer Eric’s first question about and trained to do the validation, cross-validation to be
“qualification” of the research combining these two exact. And the model can be adjusted until the prediction
areas. Apparently, we have seen papers using these two accuracy of the model satisfies us. Machine learning is a way
approaches separately. For example, we have papers that of problem-solving, which is designed in the paradigm to
use deep learning to extract features from pictures and find a solution, a recommendation system, a computation
then convert unstructured data to structured data, which system, or a classification. It is usually in a predictive
can be used in the econometric models. As a result, we mode, very computational, not necessarily in a statistical
can find out the economic meaning of those extracted way. However, it is totally different in econometrics, the
features. Therefore, this will be a valuable research. main idea of which is to develop a model that can help
Even though we can use the two methods separately, the explain the world and find out the true status of the world.
combination of both will give us an opportunity to make By hypothesis testing and other traditional social science
use of otherwise-unusable data. For example, we can put research methods, econometrics is trying to explain what
the actual data, picture data, or video data directly into is going on. In other words, econometrics explores the
econometrical models. Consequently, I perceive this as an causal effects in a statistical way. During my research, I
extremely necessary step to start. found out that machine learning usually goes along with
However, we also need to think about how to integrate design science paradigm, while econometrics always
the two approaches in an essential way, which means goes along with the economics of IS paradigm. They
technically combining them to a significant degree. It is are very different. In machine learning, you use various
not only a methodological branch but also a challenge computational techniques (some of which are periodically
for us to see the econometric sense. Although this is at computational with several layers), classification trees,
an early stage, there could be opportunities for fruitful random forest, and deep learning models, and some of
collaboration between econometrics and machine which have statistic features, such as logistic regression.
learning in the future. So the one thing for us to consider Machine learning is used as a pattern-searching tool. The
is that once you build an econometrical model, can you interesting thing is that when you develop your models,
use machine learning or deep learning as a component, you are prone to combining different models if they can
not just a separate component, but sort of an endogenous help. You may try five different models and make an
component, which means the real technological aggregation. Since all you want is achievement of great
integration for the process of data generation? This will be predictive power, there is rarely a linear relationship
a true melding of the two areas. in machine learning. However, econometrics basically
explores linear relationships. Sometimes, you can also
find a way to incorporate nonlinear ones. It is primarily
2.3 Paulo Goes a take-away from regression, with different sophistication
levels to pursue these linear relationships.
2.3.1 Idea 1: Machine learning is predictive, while To sum up, prediction of machine learning has
econometrics is explanatory nothing to do with causal influence, which is the key
point that you should keep in mind. Finding a good
First, I would like to know how many of you have prediction model with high accuracy of machine learning
used machine learning, how many of you have used is different from inferring the true underlying structure of
econometrics, and how many of you have used both or econometrics. Additionally, predictive power is different
neither? This is a very interesting topic. We are going to from explanatory power. They are two very different
discuss very different approaches, very different objects. angles.
However, they are still compatible in some way, and they
could be consistent at some level.

Unauthenticated
Download Date | 2/27/18 1:28 PM
  When Econometrics Meets Machine Learning 3

2.3.2 Idea 2: Combination of machine learning and and natural treatment, or even eliminates it. There should
econometrics is achievable and challenging be some other way for enhancement of machine learning.
Finally, the paper published in Management Information
There seem to be four ways of combining machine learning Systems (MIS) Quarterly is strongly recommended, which
with econometrics. describes the differences between predictive models and
1. The first one has been mentioned in the opening explanatory models (Shmueli & Koppius, 2011). It also
remarks already. You use a machine-learning model describes how to integrate them together. As far as I am
to feed in the variables that are going to be used in concerned, I would like to have the approach of using
an econometric model. More examples will be put machine learning first and then try to get the explanation
forward later on. of what is going on, as well as gain an understanding of
2. The second one is you have the machine-learning the underlying correlation for the co-occurrence. Then,
algorithm. Then, you can borrow some of the fun- it is time to find their causal relationships. It is believed
damental techniques from econometrics to enhance that probably, the inner loop of the hierarchical models
those of machine learning. is machine learning driven, while the outer loop is
3. Thirdly, you can do it the other way around, which econometrics driven. In my view, they are just techniques
means to borrow machine-learning techniques to and tools for us to attack all these interesting problems.
enhance econometrics models.
4. The fourth one, which is also the most interesting one,
is that you use both of them to explain the same phe- 2.4 Ramnath Chellappa
nomenon. You can take different approaches then. For
example, by doing text mining and sentiment analy- 2.4.1 Idea 1: Machine learning and econometrics are
sis, you can define different variables, and then they different in perception and understanding of problems
become part of your econometric models.
I have used some components of machine learning when
Even so, there is still one challenge. When you use machine I was an engineer. According to my experience and
learning techniques, you can always get to know the understanding of machine learning and econometrics,
precision of the predictive power in the end. For instance, the biggest difference between the two camps of people
you gain predictive power with accuracy of 85%. When you is the way they perceive problems. Econometricians are
use that in econometrics models, how can you consider basically from one side, and statisticians and computer
that prediction accuracy when you explain the error of the scientists are from another side. Take econometricians
econometric model? Maybe you do not know how to deal as an example. Since econometrics needs theories from
with that or just ignore that. As far as I am concerned, this economics, econometricians develop econometrical
is a question that should be lucubrated. Therefore, this is models according to these theories, which can be called
a common way of incorporating machine learning with theoretical models. However, with the assistance of various
econometrics. There is a great paper by Hal Varian (2014), techniques, statisticians and computer scientists use data
which is about machine learning and econometrics, and it to develop models. Take a friend of mine as an example. As
suggests that econometric models can be improved upon a statistician, he uses data to solve problems without any
by incorporating the features, including training sets, analysis of structures or backgrounds of issues. By using
validating the paradigm, and overfitting the address that bootstraps and other auxiliary techniques, he can develop
have been formed in machine learning. According to this statistical models and then find solutions for problems. To
paper, predictive accuracy can be increased by more than be specific, statisticians and computer scientists do not
50% after the integration. Econometrics helps you build a make assumptions when they develop models. They will
model without checking the adequacy of the model. As a not suppose variables to be dependent or independent
result, there is still something that needs to be incorporated, before it can be proved. The features and relationships of
such as k-fold cross-validation. Once you build a model, data have to be figured out by computer techniques, which
there will be many robust tests for evaluation, which will is called bootstrapping. Thus, computer scientists and
provide you a big table with several different columns. econometricians are totally different in their treatment of
However, you can also put the different models together data. With the help of computers, computer scientists can
and integrate them to see how they perform and how the do data processing, such as storage, analysis, organization,
big data tools can be used. Machine learning works the and classification, which can finally lead to a model.
other way around, which reduces careful, experimental, However, based on economic, econometric, and inferential

Unauthenticated
Download Date | 2/27/18 1:28 PM
4 Eric Zheng et al.

statistical foundation, econometricians always start with models with the bootstrap of machine learning. After all,
an assumed structured-stochastic model, the unknown machine learning cannot start from a vacuum, and it also
and unobservable parameters of which are estimated from needs a theoretical foundation. For example, we can use
a relevant sample by using observed data, and they can be theory for bootstrap so as to obtain good results much
used for inference and prediction (Judge, 2016). faster than when working without it.

2.4.2 Idea 2: Econometrics is theoretical foundation, and 2.5 D.J Wu

machine learning is technical assistance
Why should we integrate those two? In my opinion, we
Therefore, the way I look at it fundamentally is that one should thank Susan Athey, who has been trying to start
need a theory to put a structure on it, and then you can the theory about econometrics and machine learning.
estimate the components of machine learning, which And I also participated in a similar panel held by the
otherwise could not be done. National Bureau of Economic Research (NBER), which
As a result of the great power of computers, nowadays, was mainly about the integration of econometrics and
we can make good use of machine learning techniques. machine learning. The series of work by Susan Athey and
In econometrics, estimating the average response of the her colleagues will be a good start of reading for you to do
independent variable to the dependent variable is the main theoretical research. All the references mentioned will be
purpose of the prediction and regression process. One of great value to you, and they are worth deep reading.
goal of the model is to eliminate all the things that cannot Moreover, I encourage you to think about the underlying
really explain the dependent variables in the dependent theories and read the cutting-edge papers, whereby you
variables. For example, if you want to hide in the crowd, can make great progress.
you had better look like an average person. In other words,
the primary reason that all these methods have been used
is that they can avoid high degree of complication in 2.5.1 Idea: Both econometrics and machine learning
econometrics, and they can also be helpful to reveal the have advantages and disadvantages
commonality and relationship. Nevertheless, they ignore
and lose all the individual components in the data, which Susan Athey has already proved that there are close
machine learning can exactly address. connections between the two disciplines indeed. Currently,
Take myself as an example. Once I was asked to the integration of econometrics and machine learning
forecast the sales of music albums for a large label. As seems appealing to us. However, there is something that
usual, we ran a traditional statistical model and got nice has not been fully discovered or totally developed, and it
statistics that can fit the model well. We also did some seems to be a new opportunity for IS researchers to make
face-to-face interviews and research with some audience achievements.
and published a book thereafter. We were very confident 1. Machine learning is good at cross-validation, which
about the model and strongly believed that it is a one- can be used for model evaluation. As a model valida-
approach-fits-all model. However, the customer told us tion technique, cross-validation is used to assess the
that our prediction was wrong. As an MBA graduate, I performance of models in practice, which is really
was convinced that if I could explain 80% of the issues, hard for econometrics to accomplish.
the model would work. And I would not take into 2. Econometrics can be used to reveal the fundamen-
consideration the remaining 20%. Then, I realized that tal underlying process, which is helpful for grasping
I just provided a prediction that looked good but could the economic structure. However, machine learning
not explain everything. The remaining 20% should not cannot help contribute to the fundamentals, due
be ignored or omitted. Otherwise, the prediction will fail to which you are not satisfied. Computer scientists
in some cases. Therefore, my understanding is that even would like to resort to computers when dealing with
though the terms of econometrics may actually be used, changes and new problems. It is necessary for theo-
they depend very much on what kind of problems they are retical researchers to know about the fundamental
used to solve. Generally speaking, if there is no structured process and underlying relation, which are impor-
problem, econometrics cannot explain everything. tant for simulation, what-if analysis, and decision-
In my opinion, a successful combination of machine making effect analysis. All these can be helpful for
learning and econometrics is that you can use econometric better decision-making. Professor Yong Tan, who has

Unauthenticated
Download Date | 2/27/18 1:28 PM
  When Econometrics Meets Machine Learning 5

also joined this panel, is also trying to find out the information, which we have been doing during the past
structure, as well as to understand the fundamental few decades. Take my research with a commercial bank
underlying process, which will broaden the horizon as an example. What we found from the decision tree
of IS researchers. generated to explain the commercial loaning process for
medium-sized companies is that the loan is based on the
There are so many important problems, such as health financial attribute and the risk level of the applicant. The
care, government issues, and so on. Can we use both numbers one to five represent the risk level perceived
techniques in a new way and in a high level? There for the small- and medium-sized businesses, with the
are some theories that we do not know even if you do score one meaning the most secure and five meaning the
randomized experiments. We want to know about the riskiest levels. In classification, we use one to five as the
theories in terms of randomized experiments, and this is dependent variables, and the financial characters of the
what we want to explore. company applying for loan as the independent variables.
Consequently, machine learning not only gives you the
causal relationship and explanatory power, but also
2.6 Michael Shaw
depicts the structure, which is called tree structure. For
example, the decision-making process of the most secure
2.6.1 Idea: Econometrics and machine learning depend
company is very short. And we have a longer decision-
on each other
making process for the less-risky company. Finally,
we definitely have a very long decision-making and
My job is to give everybody a quiz. Take the bank lending
evaluation process for the riskiest company.
process as an example. Assume there are eight individuals
As a result, this process is about generating not only
with different features, such as height, income, and
knowledge that we can gain from machine learning but
education level. One person is a tall guy, with high
also the knowledge about knowledge, which is called
income and high school education. Another person is a
meta-knowledge. When you look at the tree, you can see
tall guy, with low income and undergraduate education.
characteristics about the decision tree. Therefore, you are
The third one is a tall guy, with medium income and
able to reevaluate the decision and the policy effects. In
undergraduate education. Therefore, we just want to find
summary, that is why we need econometrics. We never
out the logical relationship behind the data. Is height
know whether the tree generated or any other machine
relevant with income? Is education relevant with income?
learning model generated is stable or reliable. Therefore,
Alternatively, is there any other connection among these
we have to do validation. Over the years, we have used
features? What are the decision causes in the eight
many different techniques for validation. I think that is
individuals? What are the dependent variables? And how
where econometrics can really play an important role.
can we get a decision tree with dependent variables? It
Cross-validation is also necessary. However, the reality is
can be inferred that income is the most relevant in the
that the use of machine learning usually brings a massive
decision tree from the samples above. The tree I showed
result, which is a trigger for the training. The tree training
is not a tree example, but an acrobatic example. So what
can make the tree more understandable, which means
exactly is machine learning? It is a process that derives
you need some sort of techniques to provide a confidence
insights from a large set of data. We try to understand the
level of the validity about the machine learning model
distribution, which is a classification. By conducting a
generated. There are also some other useful techniques,
survey, we find a pattern, which is fully coincidental to
such as statistics and econometrics.
different factors or variables. Then we try to find out the
causal effect of a decision. After that, we can evaluate the
decision and its effects. So what essentially are we doing
with machine learning and econometrics? In general, we 3 Topic 2: When do econometrics and
tend to develop explanations inside knowledge through
certain data. The examples mentioned earlier are very
machine learning need each other?
simplistic, and you need to acquire more in-depth
knowledge by yourself. 3.1 Eric Zheng
When dealing with millions of data, people have
to use machine learning, data mining, as well as a Michael Shaw has just proposed a very important question.
variety of econometric models to discover valuable That is, when do machine learning and econometrics

Unauthenticated
Download Date | 2/27/18 1:28 PM
6 Eric Zheng et al.

need each other? Maybe we can have more discussions 3.3 Yong Tan
on this topic. We usually see econometrics combined
with experiments since identification is really important 3.3.1 Idea 1: Deep learning is a good approach for
for econometrics, and randomized experiments can be revealing the correlation between endogenous variables
helpful to address the identification issues. However, it and instrument variables
is still not clear why machine learning and econometrics
need each other, and when can they be helpful mutually? Discovery and explanation of endogeneity are crucial in
Let us have the panelists give us their opinions. writing an econometric paper of high quality. Although
instrument variables can be very helpful for that, it is also
quite difficult to get the instrument variables, especially
3.2 Paulo Goes when short of variables. Moreover, instrument variables
could be somewhat weak, which means the correlation
3.2.1 Idea: Machine learning and econometrics can be between the endogenous variables and instrument
beneficial to each other variables is not close. In this situation, instrument
variables are incapable of describing the variation in the
I think there may be several ways in which machine endogenous variables. Then, it is time to use deep learning
learning and econometrics could be used together. as a new method. The actual relationship between
1. Use machine learning models to feed in the com- endogenous variables and instrument variables cannot
ponents of econometric models. Just as Michael men- be conjectured or assumed easily but can be discovered
tioned, the classification results of machine learning and caught through deep learning. Consequently, deep
can be used as variables in econometric models, with learning is able to improve the predictive power of
which we can make explanation. endogenous variables and instrument variables.
2. Enhance econometric models with machine learning
techniques, or enhance machine learning models
with econometric techniques. Another approach is 3.3.2 Idea 2: Exploitation and exploration are essential
to borrow techniques from one of the two areas to to each other
enhance those of the other one. Consequently, cross-
validation, as a component of machine learning, Another example of recent research is how exploitation
is usually used when developing an econometric can help exploration. In the process of exploration, you
model. Additionally, it is possible to use econometric get out of exploitation. Knowing how exploration actually
models when dealing with overfitting in machine works and how it improves the chance of exploitation is a
learning. very complicated process that involves the methodology
3. Use both machine learning and econometrics for com- of how we actually read and understand the exploration.
prehensive understanding of a phenomenon. Some- Then finally, we can achieve exploration. Here, you need
times, there may be a tough problem, which is dif- an intermediate variable, which is unobservable. This
ficult to begin. Then you could use machine learning is the gap between where you are and where you can
for coherences, which emerge from exercises such make the exploitation. Traditionally, without assuming
as building a tree, or something similar to a decision the latent variables, it is very hard to capture the entire
rule. Then, you may have some ideas to determine the process. That is the common understanding of deep
approach to be used for the causal influence or some learning, and we can use the short-term or long-term
kind of analysis. memory to basically capture the immediate process.
Then, instead of just purely using machine learning to
Therefore, I think that after integration, they can enhance train the model parameters, because we do not have the
techniques mutually, and they also can be used in a intermediate variable, we actually use the latent variable
parallel or hierarchical way to solve the problems. to develop a tree, which comes from econometric models
at the later part. There are actually two different connected
processes from deep learning to econometrics. Then the
entire model can be achieved, as well as the estimation
of the entire parameters on both components. So in my
opinion, the methodology of deep learning can be used

Unauthenticated
Download Date | 2/27/18 1:28 PM
  When Econometrics Meets Machine Learning 7

to enrich the understanding of the generation of data, as 3.5 D.J. Wu

well as that of the whole process.
3.5.1 Idea: Why do machine learning and econometrics
need each other?
3.4 Ramnath Chellappa
From my understanding of the panel, I would like to use
3.4.1 Idea: We need to know both what and why the reducible version. The reducible version is to use the
computer to write a paper and be accepted by Information
With the abundance of theoretical knowledge and the Systems Research (ISR). That means using machine
mastery over causal relationships and effects, you will learning to generate models in a systematic way. Machine
find econometric models to be a good start. Moreover, if learning is very good at generating models systematically.
the past data generation is consistent with the future data However, it is uncertain whether the cause of inference
generation, the prediction has been proved to be extremely can be maintained, as Paulo just emphasized. Since each
good, which is perfect. However, it is not always the case. of the two disciplines has strength in models and inference
That is the reason why your prediction is not always so separately, they need each other most of the time.
good compared to what the model explains. There are Can we use machine learning to generate models
basically two fundamental reasons for that. The first one systematically while maintaining the causal inference?
is that the data you are using to generate the predictive Nobody knows the answer, and it is a new challenge, which
model is not consistent with the future data generation. you are encouraged to take. Another interesting example
The second reason is that the theory does not apply. is to use machine learning to mimic the decision process
On the one hand, when the underlying process is of the judges. Sometimes, we think it is a little bit too much
unperceived, machine learning can help with identification to solve the underlying societal problems. For example,
and recognition, and it also can be useful in the process in the US, many people are potential crime suspects. The
of theory building potentially. If you want to explain judges have to decide who should go to trial. Imagine, the
why it is, you have to tell what it is first. Something like majority of the people ending up in trial have been found
when Amazon says people who like this would also like to be innocent. It would have been a waste of resources if
something else, there is no reason for why, there is no the person had been kept in prison earlier. Therefore, the
explanation for it is what it is. judge is trying to use intuition, experience, and all that
Basically, this is just what we happened to observe kind of stuff in the trial process. This research involves the
in data. When doing explanation, people always use the use of machine learning to mimic the judge, instead of the
theory, which contains many constraints. For instance, outcomes of the judge, such as their emotion, their eyes,
when doing a movie classification, people assume that and many other aspects. Then, a decision is made about
thriller movies are similar to horrible movies. This may who is going to be devoiced. All you have to do is to use
come from some kind of category, which machine learning machine learning to learn, to train the observables. And
techniques are good at and can be of great help. On the then, you use a robot or machine for a better decision.
other hand, as I have mentioned, econometrics can be When you challenge yourself with these theory problems,
helpful to machine learning with the quick converging you can finally solve some societal problems. In China,
of process. When the processes are based on economic there are also many issues that can be explored, such
theories, econometrics helps the process converge quickly, as health applications, government issues, and so on.
which is followed by good identification and bootstrap. Consequently, there are many opportunities for you to
The last thing I want to explain here is what D.J Wu make an impact.
has mentioned just now. Since more and more machine
learning methods and certain practices have been inducted
into traditional econometrics, such as the out-sample 3.6 Michael Shaw
indexing in the holdout, some sort of prediction seems to
be coming. Although you have a prediction model, it does 3.6.1 Idea: A cocktail approach can be used in the
not mean that you can explain everything, even if your combination of machine learning and econometrics
model has a high level of prediction accuracy. Prediction
is not only to do with machine learning. Increasingly, In modern medicine, there is an approach called the
because of the use of all-sample indexing and holdout cocktail method, which has been used to make quite a
samples, as well as doing for particular purposes, it seems few breakthroughs. This kind of approach combines two
to gain some increasing attraction. or more techniques for a treatment, which contributes
Unauthenticated
Download Date | 2/27/18 1:28 PM
8 Eric Zheng et al.

significantly to applicable results. The way we combine 4 Topic 3: How to get prepared for
machine learning and econometrics is just like a cocktail
approach from different disciplinary perspectives. It is not
the combination of econometrics and
necessary to emphasize that they are not equal because machine learning as researchers?
they originate from different research foundations.
Econometrics seems familiar to us since it has been
4.1 Eric Zheng
around for a while. Machine learning comes from a
tradition that started from the so-called symbolic pattern
In this panel, we have not only the panelists, but also a
recognition, and it has already combined with statistical
great group of editors. On behalf of the audience, I want to
pattern recognition. Both engineers and scientists want
ask them what they would like to recommend to authors
to solve problems as much as possible. It should not be
here regarding how they could prepare themselves for
overemphasized that these are two different approaches.
research that combines econometrics with machine
Let us look at the problem at hand and try to solve the
learning. What they should and should not do?
problems with reliable results that can validate theoretical
hierarchical hypotheses, thus laying out the path for
important scientific discovery in this way. I always think
4.2 Olivia Sheng
of them as research approaches for whatever research end
I am trying to reach. We should also notice the pros and
4.2.1 Idea: It is necessary to master domain knowledge
cons of different approaches. In this case, econometrics
and use tools skillfully
obviously has a strong foundation that we are all familiar
with. What will be the foundation and benefits of using
In my opinion, econometrics and machine learning are
machine learning? Specifically, with symbolic processing
just methods, and their routes are always joined together
capabilities, machine learning gives you more structured
with statistics upstream. Within computer science, there is
information, as well as texture information. You know
inductive learning. Besides, there are predictive methods,
a little bit about the content, and perhaps it provides
or methods that can make predictions or discover some
explanation from a different angle. Consequently, the
common patterns that might be taught in data mining
combination of the two can be quite useful. For example,
classes and machine learning classes, as well as statistical
you want to provide a decision maker, no matter what
methods. Statistics takes one direction and machine
the field is, with the result of your analysis. With pure
learning takes one direction, while deep learning handles
econometrics, most of the time in statistics, the base is a
large scales of data in the process. Here, I have some advice
numerical base. With machine learning, you can provide
and then I will come back with the question. We can use
explanation from many different angles, and you are able
certain tools that we have trained and that can be applied
to provide the insightful information for the decision
better than the tools that we do not know.
maker involved. I do think that the current high-level
If you truly care about the benefits of combining
machine learning can prevent the shortcoming. Once it
these two, you must know the tools very well. Since I am
is actually used in any industry, you can generate some
trained in machine learning, I have to learn econometrics
decision by scientists, and then the decision will lead to
and many other tools that remain unknown to me. When
other decisions. It does not converge to a visible solution,
I apply econometrics to deal with problems, I need to be
but explores because of the unpredictable results that
very cautious. As a serious researcher doing practical
nobody can explain, which would cause problems.
research, one has to master the tools, as well as the usage
As a result, I think using different approaches to help
and application methods. Additionally, it is necessary
each other out and then understanding the underlying
to compare them with other tools and find out the
decision-making process is important, not just relying on
similarities. All of these are challenging, and I can only
the predictive ability or the explanatory power. Moreover,
probably comment on that. When you solve a predictive
I do appreciate such a combination of various approaches
analysis problem, you either use the machine learning
where you can be satisfied with the feeling that it is a very
method or use the econometrics method combined
nice approach.
with cross-validation; you either evaluate model fit or
evaluate predictive errors: all that depends on your goal.
Different communities have different methods that can
predict better. My expectation is that predictive analysis

Unauthenticated
Download Date | 2/27/18 1:28 PM
  When Econometrics Meets Machine Learning 9

researchers from a machine learning background have the machine learning does not provide explanations; machine
attitude of improving performance. However, the field has learning models do provide explanations, they provide
evolved very long, and getting good performance is not explanations through logic, you know, through logic or
an easy task. If you want to have good performance, you business domain knowledge. Therefore, you can actually
need to know the problem very well, as well as the data, provide good explanations. The question is whether you
the method, and the process. It is necessary for you to do can interpret them with theoretical explanations. When
numerous evaluations and try different things. you are using them, it is important whether or not you
I will take the decision tree as an example. Data-driven can explain why and what it is. One of the things related
method already gave me the highest evaluation. If I come to the pruning of trees I want to mention is that it gives
from a theoretical background, for credit or for voting, then every other exception. It provides opportunities for
I will develop a model and hypothesis that are only related explanations. The branches that you do can be potentially
to income and so on. These two ways may find the same good opportunities for explanations. They may not have
result. You can use these to do cross-validation very easily. some statistical relation, such as those of the sample,
So my suggestion is that we can no longer get away with but they do provide you some insight that might be some
good performance, we have to explain. It does not mean potential explanations of certain things and it might
that all of us need to be very knowledgeable with behavior be some interesting research opportunities. From my
theory, but at least you have to possess some good domain perspective, I hope you do all kinds of methodologies.
knowledge, which can help us explain well. I will not stop
immediately after I have good performance, and I will go Acknowledgments: We would like to acknowledge
out and find some reasons to help explain. Today, I am Professor Qinghua Zhu from Nanjing University of China,
paid to do so because I am a consultant, I definitely have Professor Ming Fan from University of Washington of
to find some explanation because your stakeholders or the USA, Professor Zhenhui (Jack) Jiang from National
your clients ask you that your deep learning predicts so University of Singapore and Professor Han Zhang from
well and why, and whether it can predict better and have Georgia Institute of Technology of the USA for their great
hierarchical correlation relationship. And benchmark support to this paper.
well. As far as I know, the skill of the problems and the
skill of the applications are so big that we cannot explain
everything about them. We can just explain some features.
However, there are so many analyses and predictions that
References
we need to explain. Therefore, there is not only predictive Judge, G. (2016). Some Comments on the Current State of
analytics, but we also need to use all kinds of training Econometrics. Annual Review of Resource Economics, 8(1), 1.
and methodological backgrounds to help us develop good Shmueli, Galit, & Koppius, O. (2010). Predictive analytics in
explanations. information systems research. Social Science Electronic
Publishing, 35(3), 553-572.
Susan Athey. (n.d.). Retrieved December 27, 2017, from: https://
www.gsb.stanford.edu/faculty-research/faculty/susan-athey
4.3 Alok Gupta Varian, H. R. (2014). Big Data: New Tricks for Econometrics.
Journal of Economic Perspectives, 28(2), 3–28. https://doi.
4.3.1 Idea: It is important to choose appropriate org/10.1257/jep.28.2.3
methodologies for problems

From my perspective, it is very important to choose the

appropriate methodology for the problem. This panel
has made some good observations about where they
can be enhanced, the gaps that can be filled, and the
challenges that econometrics techniques present and
that can be overcome. You will see papers that actually
do interesting things: they do not actually include these
things in the modeling, but when they are looking at or
doing the estimations, they show all the explanations.
So if you cannot do it theoretically, you can use machine
learning techniques, because in my view, it is not true that

Unauthenticated
Download Date | 2/27/18 1:28 PM