Machine learning
Why did rectified linear units largely replace sigmoids in deep networks?
AdvancedMultiple choice
- AA sigmoid saturates, so its gradient vanishes and deep stacks stop learning
- BRectifiers are smoother, which helps the optimiser
- CRectifiers bound the activations, which stabilises training
- DRectifiers guarantee convexity of the loss
The worked solution is in Premium
The answer, the full working and the one idea to take away – for this and all 1,322 questions in the bank. Answer it in practice and your working is marked, with a known mistake named when you make one.
Report an issue with this question
Of the kind reported at
More machine learning questions
- A softmax output layer receives logits (2,1,0).Foundation · Premium
- A batch normalisation layer follows a fully connected layer of 128 units.Applied · Premium
- A convolutional layer uses 3 × 3 kernels, 16 input channels and 32 output…Applied · Premium
- How many learnable parameters does a fully connected layer from 100 inputs to…Foundation · Premium
- Why is ordinary shuffled k-fold cross-validation wrong on a financial time series?Applied · Free solution
- What does embargoing do that purging does not?Advanced · Free solution