Yes, it was established long ago that James was married to his work with no time for a family or kids. Key words being long ago.
I sometimes forget people can't see into my head and have no idea the changes these characters have gone through. While most of them are spoilers I have no intention of sharing, I can say that as a successful man in his 30-somethings living during the 1650's – 1730's at the height of piracy, James would definitely have a wife and more than a few kids running around back in England.
He loves them dearly, and when he's not babysitting in the Caribbean, tries to see them when he can.
Characters and Artwork © Shamine Athena King