James added 14 assists and nine rebounds for the Cavaliers, who have won their last three games.
Kyrie Irving earned 24 points while Reservist Channing Frye shone with 21 points.
Frye managed five long-range shots, including a crucial one that gave the defending champions 117-115 to the defending champions, with 59 seconds on the dial.
The Raptors seemed in good shape after two straight shots by Terrence Ross at the end of the game. They then led by six points, with five minutes to dispute, but the Cavaliers had the last word.
Kyle Lowry dominated with 28 points for the Raptors, who will be back in action as of Wednesday, receiving Golden State.