Open
Description
Description
Having dimension-wise reduction is important for building many neural network layers like RNNs, or for performing pooling operations in CNNs.
Implementation Ideas
- A new
Function
variant, sayReduce(dim, operation)
, that merges or removes one dimension from the shape. - Tie into derivative logic if needed (e.g. derivative of a sum is straightforward, but derivative of a max might require special subgrad logic).
Next Steps
- Implement a
FunctionInfo
that handles reduce ops. - Ensure that the resulting shape and derivative edges are consistent with the rest of the library.