The median is a very extreme way to drop outliers. In general, I think a “centered mean” does a better job. For example: take the mean of the middle three-quintiles: if 10 games were played then drop the two best and two worst and take the mean of the rest.

This also gets rid of the worst outliers and also prevents big swings that you could potentially get from medians on say: 322 / 394 / (400) / 505 / 520 if the next game score is 300, the effect on the median is small (-3). If the next one is 500 instead the effect is very large (+50) even though both are the same difference from the median of the previous scores.