Neural Networks in Brief
"Explain how a neural network works, and tell me what goes wrong on financial data." A complete answer covers what the network computes, how its gradient is found, how many parameters it has against the number of independent observations, and the ways a backtest of one can mislead.
What a network computes
A feed-forward network is a chain of layers. Each layer applies a linear map, , and then a non-linear activation to each element, most commonly the ReLU, . The output of one layer is the input to the next, and the last layer produces the prediction.
The activation is what gives the network its power. Without it, a stack of linear maps is itself one linear map, however many layers there are, and the network is just linear regression. With it, a network with one hidden layer of enough units can approximate any continuous function on a bounded range of inputs to any accuracy. That result says a network can represent the truth. It says nothing about whether it can learn it from the data available.
The rest of this lesson is for subscribers
Unlock every lesson in Machine Learning for Quantitative Research, and every other premium course.
Subscribe to continueTest your knowledge
Keep reading Machine Learning for Quantitative Research
27 lessons in this course, and every other premium course, on one subscription.
- Every lesson in every course, with the worked examples and interactive simulators
- Graded questions on every lesson, with explanations for the wrong answers as well as the right one
- The trainers, timed assessments and brainteaser library that go with them