On a different note,
#Add the episode to the experience buffer
bufferArray = np.array(episodeBuffer)
episodeBuffer = zip(bufferArray)
In the above block, do you need the pre-processing of episodeBuffer (first 2 lines) before putting into myBuffer?
For me, just myBuffer.add(episodeBuffer) without the first 2 lines seems to work just fine…
Do you have some performance gain if you do the pre-processing ?