Comey actually explained a contact with Trump that may explain why Trump was happy to meet alone with him for sensitive conversations.
Comey testified that at their first meeting when Trump was the president elect he told everyone to leave the room and they discussed in private the now famous dossier. Trump may have got the impression from that first encounter where Comey pushed everyone out of the room so he could talk privately with Trump that this was acceptable behavior and contact between him and Comey.
How would Trump see it otherwise? Trump actually has always preferred 1 on 1 meetings his entire life so from day 1 that was demonstrated as proper from Comey himself, hard for Comey to cry about it after the fact when he was the guy who established that standard.
I find it beyond crazy for Comey to claim he was scared and a coward and too timid to stand up to Trump and point out that any conversation he felt was wrong should not happen. This is a man with a lifetime of working in the law, the DOJ, the FBI and he was too scared to stand up for himself and the office he served? I do not believe for a second he was scared of Trump, but if true he should never have been the director of the FBI. That office requires a backbone, and he just admitted he had none.