I think the serpent-men are the "first" version of the pilgrims. They have the same body structure, the only difference is that Pilgrims clothes have a complete hood and are dark.
Maybe the first prayer of dragons tried to became dragons, and they failed upon their greed.
Some of them turn their back to the dragons, in want of answers, and finnaly followed Londor. They found another way to ascend, with that thing they have on their back, and that was certainly found by clerics.
Maybe those firts clerics were from the way of white, and as Carim ascended and finnally destroyed the way of white, they took this little gift on the way.
Londor is somehow related to Carim: Their miracles (because they only use miracles) where once those of Velka, and Friede (who's one of the 3 sisters of Londor) give you the Chillbite ring, and those kind of rings come from Carim.
After taking the way to become angels, the Pilgrims tried to ascend through Hollowing, and many of them failed, and tried to rebirth thanks to Rosaria. Those who rebirthed too much finnaly became man gru, and this is why you find one in the Lothric Castle near the cage with the Divine pillar of light.
Maybe they understood it was a failure, and forecast Rosaria in the cleansing chappel, and all her followers did the same.
But some succeed before that, and became the angels that made the faith in them true in Lothric.
The biltrim butterfly are just the rotten version of those angels, maybe far after their "true" body died, and just wandered where they were.
Also, remember that there were Serpent-men after Oceiros bonfire!
They all are here, in the Lothric castle, the man gru, the serpent men, the pilgrims and the angels.
Also, Londor was found thanks to Kaath, a primmordial serpent., and you can find his statue in Lothric castle, just before the grand archives. Maybe the "primmordial" is not because they were here at start of the world, but maybe because they are the first of their kind: The first serpents?
They mostly looks like followers in dark souls 1, Fraampt followed Gwyn, but maybe Kaath followed his counter part, Velka?
Velka is related to Occult, which is the "god slayer" power of Dark soul 1, and also the dark related infusion.
Velka was considered as a heretic, and Londor is also considered as heretics. They both fought for men, Velka guiding you with her crows and cleaning your sins that the Blades of the darkmoon were hunting, and Londor try to make the men ascend to the throne.
And the more we go, the less influence the darkmoon blades have!
Look at what the covenant became compared to dark souls 1: No sin to hunt, no warriors to follow. Her "captain" is lost in a tower with no way to go in.
And finnaly, the way of blue is replacing it: Maybe they are the human counter part of the Darkmoon blades? Not hunting sin, but mostly, protecting peoples from invasions?
An other point upon Velka being related to Londor, is that Londor's sisters were know as great swordmen, and the Velka's rapier from dark souls 1 says "A symbolic, powerful thrusting sword used by the pardoner serving Velka, Goddess of sin. It is no mere symbol to be sure; the pardoner is an inhuman swordsman, and wields this enchanted blade with special sword technique."
They both were known as incredible swordmen.