John wrote the lyrics. Paul has never denied that, other than mentioning he *may have* helped lyrically slightly during the middle 8 section. I've never read anywhere of Harrison having any involvement. The inspiration for Lennon to write such a nostalgic lyric actually came from a reporter. Lennon later remarked in one of his last interviews the following:

"It was sparked by a remark a journalist and writer in England made after In His Own Write came out. I think ‘In My Life’ was after In His Own Write… But he said to me, ‘Why don’t you put some of the way you write in the book, as it were, in the songs? Or why don’t you put something about your childhood into the songs?’

The original lyric sheet mentions Penny Lane, and reads more as a poem than a lyric. You can see an image of his handwritten first draft here: https://www.reddit.com/r/beatles/comments/68vnd0/john_lennons_draft_lyrics_for_in_my_life/