Roan GylberthPemilu 2024: Post-Election AnalysisHasil, Pengalaman, dan Pelajaran5 min read·Feb 19, 2024--1--1
Roan GylberthinKonvergen.AIUnderstanding DropoutOne particular layers that are useful, yet mysterious when training neural networks is Dropout. Dropout is created as a regularization…6 min read·Jul 21, 2019--2--2
Roan GylberthinKonvergen.AICross-entropy and Maximum Likelihood EstimationSo, we are on our way to train a neural network model for classification. We design our network depth, the activation function, set all…5 min read·Feb 16, 2019--7--7
Roan GylberthinKonvergen.AIFinding the Minimum: Escaping Saddle PointsWhen I did my weekly Arxiv chores (via arxiv-sanity.com, which is a great site! Thanks Andrej Karpathy), I stumbled across this paper by Du…4 min read·Nov 18, 2018----
Roan GylberthinKonvergen.AIAdaptive Method Based on Exponential Moving Averages with Guaranteed Convergence ; AMSGrad and…From all optimization algorithms that we have covered in this series, Adam and RMSProp are very popular among Deep Learning practitioner…6 min read·Jul 21, 2018----
Roan GylberthinKonvergen.AIModifying Adam to use Nesterov Accelerated Gradients: Nesterov-accelerated Adaptive Moment…On the previous post, we have discussed Adam algorithm, which could be seen as the way to combine the advantage of Momentum method and…4 min read·Jun 24, 2018----
Roan GylberthinKonvergen.AIAccelerating the Adaptive Methods; RMSProp+Momentum and AdamOur last post discussed two other adaptive algorithms which are an extension to the Adagrad algorithm, i.e., Adadelta and RMSProp. As we…4 min read·May 21, 2018--1--1
Roan GylberthinKonvergen.AIContinuing on Adaptive Method: ADADELTA and RMSPropIn our last post, we have discussed the difficulties of setting the learning rate hyper-parameter, which can be mitigated by using the…6 min read·May 4, 2018----
Roan GylberthinKonvergen.AIAn Introduction to AdaGradWe have discussed several algorithms in the last two posts, and there is a hyper-parameter that used in all algorithms, i.e., the learning…5 min read·May 3, 2018--4--4