Great write up.

Regarding your improvement ideas you could start searching for lines based on where the lines were in the previous frame. Furthermore setting a max amount of change regarding angle and coordinates could help to ignore false detections. Another idea would be to average the results of 2 or 3 frames to reduce “jumping” of lines between frames. Also since you’re exclusively interested in edges going along a certain direction a custom sobel-inspired kernel could go along way.

Just a couple of thoughts ☺