You haven't had sex with him. However, you do exchange numerous emails a day, talk on the phone several times a week, discuss your marital relationship with him and every so often wonder why you didn't hook up when you two were in high school.
No sex, just a special friendship that you keep to yourself.
No sex, just a confidant, someone who understands your problems.
No sex, just long conversations on the phone, chats online and cozy lunch dates.
No sex, just a connection on Facebook with an old high school flame.
No sex so, what is the harm?