@Radagast
Heinlein's Lazarus Long, going back in time to impregnate his mother with himself.
That is incorrect.
While Lazarus Long DID go back in time and have sex with Maureen Johnson Smith, she was ALREADY pregnant at the time. During their first attempt to have sex - at the time, in or using the backseat from the Ford he was driving - they were foiled because a young Woodrow Wilson Smith was IN the backseat, because Sergeant Ted Bronson had promised to take him to Electric Park.
Woodrow Wilson Smith later grew up to be Lazarus Long, who used the name Ted Bronson during his time travel visit to Kansas.
During a previous engagement in the Starship Dora, Lazarus had sex with Lapis Lazuli Long and Lorelei Lee Long, who were female clones that were his genetic duplicates, as the geneticists duplicated his X chromosome so they'd be female. The discussion was that it would effectively be masturbation at worse, and not incest (and in his case, due to having a clean gene chart, no issues with progeny should he impregnate them).
Thus, your time traveling man having sex with himself - which was written about by Heinlein in 'All You Zombies' and Gerrold in 'The Man Who Folded Himself,' among others, would be masturbating.
ETA: I just realized something scary - I did ZERO research before writing this response, because I already KNOW all of this stuff. How many times did I read those books?