Computer Science VS statistics

I think that there is a fundamental difference between statistics and much of current computer science type machine learning.

In statistics we rely on the evidence and tend to factor out as much bias as possible.

In machine learning in the form of neural networks, genetic algorithms etc we sort of encourage bias. We are saying that the representations as weights of connections in a network is a good way of representing what is to be learnt. This may or may not be the case.

So if it is the case then ANN will do well but if it is not then ANN will do badly.

For me it is like having two civilisations, one which has perfected working with circles and attempting to approximate everything with circles. The other civilisation works with straight lines. Everything to it is a line. The two civilisations will each have good ways of tackling problems but each has its own focus. There is no reason to suppose that they will develop in the same way.

To a hammer, everything is a nail and to a wrench everything is a nut/bolt.

It's both. ML would not have gotten where it is without both. And, unfortunately, there are those from stats and CS that don't realize the value of the other camp.

Without statistics, a lot of today's technology would not be possible. And most ML algorithms have heavy statistical theory backing them up. Without CS, though, the ability to apply those algorithms would be almost non-existent.

The two camps have a lot to learn from each other. ML researchers from CS need to stop re-inventing the wheel and acknowledge the use of statistics. Statisticians have to acknowledge that they dropped the ball, and instead of riding the data science wave like they should be, they're being perceived as backwards. They have to advocate for the use of statistical thought. (Because, well, it's necessary.)

There are some from both camps doing this, but, from where I stand, not nearly enough.

From other answer before merge:

Without computer science, there would be no real ideas on AI.

Without statistics, there would likely be no computers, as statistics was used at the core of a lot of scientific discoveries. Then again, without computers, statistics would be done by hand... not a welcoming prospect.

Without math, we'd not have even gotten cities built.

It all builds on and into each other.

Statistics is data analysis founded on probability theory (well, depends on who you ask, but that's fairly standard). There are a lot of mathematical concepts that exist that are, while not based in probability or statistics theory, were motivated by a need for them in solving data problems.

More and more I'm finding divisions like this rather arbitrary. Who cares whether this technique is 10% math, 20% stats, and 70% CS? CS has huge roots in math. So does statistics. If an algorithm can be viewed statistically, shouldn't it be considered from that viewpoint as well as the other viewpoints? More ways of looking at a problem lead to better solutions.

Why limit your viewpoint? I think we do this implicitly in our own minds by imagining such differences between the fields instead using what works when it's appropriate.

I don't think it makes sense to partition machine learning into computer science and statistics. Computer scientists invented the name machine learning, and it's part of computer science, so in that sense it's 100% computer science. But the content of machine learning is making predictions from data. People in other fields, including statisticians, do that too.  It is more that computer scientists and statisticians view "making predictions from data" through different lenses. Here are some stereotypes, which I am adding as a header so I don't have to say "tend to" and "mostly" everywhere.

Computer scientists view machine learning as "algorithms for making good predictions." Unlike statisticians, computer scientists are interested in the efficiency of the algorithms and often blur the distinction between the model and how the model is fit. Computer scientists are not too interested in how we got the data or in models as representations of some underlying truth. For them, machine learning is black boxes making predictions. And computer science has for the most part dominated statistics when it comes to making good predictions.

Statisticians are concerned with abstract probability models and don't like to think about how they are fit (ummm, is it iteratively reweighted least squares?). Statisticians pay more attention to interpreting models (e.g. looking at coefficients) and attach meaning to statistical tests about the model structure. Computer scientists might reasonably ask if statisticians understand things so well, why are their predictions so bad? But I digress. Unlike computer scientists, statisticians understand that it matters how data is collected, that samples can be biased, that rows of data need not be independent, that measurements can be censored or truncated. These issues, which are sometimes very important, can be addressed with the probability-model approach statisticians favor.

Computer scientists and statisticians both ignore questions of causality when they build models. Right now causation doesn't play much of a role in "machine learning," even though it obviously matters for making predictions. Economists are better about acknowledging this. Maybe someday there will be a future version of this question that will mention causal modeling as a third aspect of machine learning.