The name, "Jesus Christ", itself also became a rich source for such symbols. These became known as Christograms or divine monograms and they are actually quite common, though I doubt that most people are aware of what they are let alone that they even exist. They appear on priests' and ministers' vestments, on banners in churches, and in inscriptions and paintings. For example, in the coronation scene in Netflix' The Crown we see the "JHS" Christogram on the back of the Archbishop of Canterbury's vestment (see below).
Of course, since the New Testament was written in Greek, those Christograms are based on Greek letters, though occasionally with some transliterations to Roman letters (what modern English uses). In Greek, "Jesus Christ" is written in all caps as:
In mixed case, it is written as:ΙΗΣΟΥΣ ΧΡΙΣΤΟΣ
The creation of Christograms draw from both ways. Spoiler alert: If you have a sharp eye, you would notice that the capital eta ("Η") looks very much like a Roman "H" and so you should already know where this is heading.Ιησους Χριστος
An early Christogram is the Chi-Rho from the first two letters of "ΧΡΙΣΤΟΣ" ("Christ"). It is first that I had noticed only because I spotted it after having studied Greek in college. It also seems to be the most common one at present (at least in my experience) and is to be found in both Catholic and Protestant churches.
It is formed by superposing the first two letters of ΧΡΙΣΤΟΣ, the chi and the rho, over each other, though the chi is made smaller or more commonly the rho is elongated so that the chi will not obscure the rho -- 
.
Now that you've seen it, I'm sure you can remember having seen it before. Or at least now you will start seeing it in churches.
Another early Christogram is the IX monogram, which takes the first letters of both words, the iota from Ιησους and the chi from Χριστος, and superimposes them over each other thus: 
.
Here is an example from a late 3rd-century Constantinople sarcophagus:
Another one is the IH monogram which takes the first two capital letters in ΙΗΣΟΥΣ and superimposes them over each other thus: 
.
I'm sure that the similarities between this Christogram and the company logo for
International Harvester are purely coincidental.
And then there's a family of Christograms that actually play a part in putting the "H" into "Jesus H. Christ".
These play on the first two and the final letters (or the first three letters, depending on how you think of it) of Ιησους. The Christogram that I saw in Cádiz, "JHC", falls into this category as does the one on the vestments of the Archbishop of Canterbury in The Crown, "JHS". This family is also known as IHS. Here are some examples, the first one modern and the second medieval:
So why the variations?
The "J" is simple, because in Latin the "I" and the "J" were virtually interchangeable, especially when followed by another vowel in which case it became a palatal sound very similar to the initial "Y" in English (eg, yard, yellow, Yiddish, yoke, Yule). The "J" in German still behaves in the same manner, ja? So by Latinizing the iota ("Ι"), it could become a "J". Simple. Indeed, that's part of how the name of Ιησους got changed to Jesus -- though yet again the German pronounciation of "Jesu" is much closer to the original Koine Greek.
BTW, the interchangeability of "I" and "J" was used as a minor plot point in Indiana Jones and the Last Crusade (1989) where Indy has to spell out "Iehova" and mistakenly starts at first with "J" and nearly falls through the floor as a result.
The Σ becoming an "S" or a "C" is a bit more involved. The "S" is simple enough, since it's just a transliteration from Greek to Roman, but the "C" needs a little more explaining. Turn your attention to the final sigma, "ς". In that form you can see the makings of the Roman "S", albeit proportioned differently. But often that lower "tail" would disappear in which case it would look like the Roman "c" (also used in English). That is why I talked of using the first two and the final letters of Ιησους, because the final sigma looks very much like a "c".
But it turns out that there's more to it than that. In the process of researching for this page, I learned for the first time of the "lunate sigma." It turns out that in the Hellenistic period (4th and 3rd centuries BCE), the Σ used for carving inscriptions was simplified to a C-like shape. That form was called "lunate" because of its crescent shape like a crescent moon. The Cyrillic alphabet used by Russian and other Slavic languages was based primarily on the Greek alphabet; the Cyrillic letter "C" for their "s" sound appears to have come from this lunate sigma.
So this IHS/JHC family of Christograms range in form from being all Greek to one Roman and two Greek letters to two Roman letters and one Greek. But in each variation, the middle initial, the eta ("Η"), remains Greek.
In researching for this page, I also learned that it's not just a coincidence that the eta, "Η", looks like the English "H", but rather yet again there's a bit more to it. According to Wikipedia (H), both Greek and Latin had gotten the letters "Η" and "H", respectively, from the Phoenicians who had gotten it from earlier sources. The Phoenicians used it to represent a sound like the English "H". At first both Greek and Latin used their versions of that borrowed letter for the same sound, like the English "H", but then in Greek its sound changed into the vowel sound we know from Ancient and Koine Greek.
As a sidenote in case you know a little about Russian, the Cyrillic letter, "H" for the "n" sound, is not related to the Greek Η. Rather it started as a borrowing of the "nu" ("Ν"), which looks exactly like the Roman "N". At some point in the process of that borrowing into Cyrillics, the end points of the cross member of the "Ν" was migrating closer to the vertical center and the cross member was becoming more horizontal. The Cyrillic "H" just represents the end result of that process which was just a temporary thing in Greek.
So returning to our thesis, it appears rather likely that the "H" in "Jesus H. Christ" originated from the IHC family of Christograms, especially the one that I had seen in Cádiz, Spain, "JHC", which on the surface appears to be somebody's initials, namely those of "Jesus H. Christ".