Those who forswore them
Quicksilver and the Scarlet Witch were not the only ones to abandon one side for another. For the most part, Xavier was able to successfully indoctrinate his disciples with his dream. However, for everyone there is a breaking point. For Piotr Nikolievitch Rasputin - better known as Colossus of the X-Men - that point had come. After the murder of his parents, followed by his little sister - the last of his family - dying of a virus, which attacked only mutants, Colossus knew that something needed to change. Clearly, he had tried Xavier’s way and the only result was that he had lost his family. [X-Men (2nd series) #17, Uncanny X-Men #303]
Thus, when Magneto appeared before the X-Men at his sister’s funeral, Colossus chose a different path. Colossus would never be fully converted to Magneto’s vision but it took him a long time to reject it fully and return to Xavier’s side. And it was clear that he would never be exactly the same. [Uncanny X-Men #304]
For Magneto, it is almost difficult to point to a follower that moved to Xavier’s side - as few mutants stayed with the self-styled Master of Magnetism long enough to learn at his knee. One group that did, however, was a set of ardent followers who declared themselves his Acolytes - led by the maniacal Fabian Cortez. The Acolytes sought out Magneto, practically begging to fall to their knees in his service. However, they seemed to learn very quickly that they preferred following in his memory to following him in the flesh. [X-Men (2nd series) #1-3]
However, for the harshest example of abandonment of Magneto’s cause came from Quicksilver and the Scarlet Witch. When he first recruited them, Magneto had no clue that, in truth, the two were his children - a fact that they learned years later. Still, rather than having this endearing them to him, it disgusted them both. Over the years, the Scarlet Witch had continued her membership with the Avengers, with occasional sabbaticals for personal relationship and a new family she had started.
Likewise, Quicksilver had joined the Avengers and, later, the government strike team known as X-Factor. It was during his tenure with this group that he came across the Acolytes and their leader, Fabian Cortez. At the time believing Magneto dead, Cortez wished to have the son of the man they worshiped join and lead them, only to have Quicksilver reject him, stating that they was nothing of his father’s legacy that he wanted a part of. To be rejected ideologically is one thing. To have your memory be spat upon by your children is the ultimate abandonment. [X-Factor (1st series) #92]
Irony, however, had not completed its duties. When she first came to the attention of the X-Men, Lorna Dane - the mutant eventually known as Polaris - had been tricked into believing that Magneto - in actually a robot duplicate of Magneto - was her father. Years later, while working with him on the island nation of Genosha, which Magneto had recently come to rule, Lorna stayed by his side, helping him run the fledgling nation. During this time she learned in truth that she was his daughter, but before she could confront Magneto with this knowledge, Genosha was destroyed and its citizens annihilated by the monstrous Sentinels of Cassandra Nova.
Being present, Lorna witnessed firsthand the murder of over sixteen million mutants - and barely survived with her sanity intact. Even when found alive months later by Xavier, Lorna ranted and raved incoherently. 
Though she quickly found her way back to mental health, Lorna was changed. She had previously been an X-Man and worked for mutant rights as both a member of X-Factor and using that position to make public speeches. In all of these roles, she had been a fervent supporter of Xavier and his dream. Now, however, her position had changed dramatically. Visiting the site of the Genoshan holocaust with Xavier, she explained her thoughts. [New X-Men (1st series) #132, Uncanny X-Men 442-443]
Forsaking their own dream
A crisis of faith is always possible - even by those who began such a crusade. Both Magneto and Xavier have had such moments, albeit brief on both sides. For Magneto, it was a genuine attempt to try a path he had rejected decades before - realizing that his own way was not working. For Xavier, it was a moment of uncontrollable anger, which would have deadly consequences later.
During one of his Machiavellian plans, Magneto had created a machine which he could use to unleash Earth’s magnetic field against the nations of the world. As they had always done in the past, the X-Men arose to oppose Magneto, tracking him to his island headquarters. While Magneto engaged the elder X-Men in combat, Shadowcat used her phasing powers to disrupt an irreplaceable computer memory core. Incensed at the waste, Magneto attacked Kitty, apparently killing her. Shocked back to his senses, Magneto finally, after decades of maniacal actions, realized what a monster he had become. [Uncanny X-Men #150]
Over the following months, Magneto reconsidered his ways and eventually even surrendered to government forces to stand trial for his crimes against humanity. The trial was halted when it was attached by the mutant children of Baron von Strucker, who sought revenge against what Magneto had stolen from their father so long ago. The destruction of their attack was abated by Magneto’s defense of those attending - and irony not lost on those he saved.
However, subsequent to the attack, Magneto realized that Xavier was dying. Xavier’s deteriorating condition was not a result of the Struckers’ attack, but a mugging that had happened weeks before. Barely able to speak, Xavier made his old friend promise to carry on in his place. Reluctantly, Magneto agreed. [Uncanny X-Men #199-200]
As it turned out, Xavier did not die. He was saved by the arrival of his lover, the Shi’Ar princess, Lilandra, who had detected Xavier’s condition via their telepathic link. Though Xavier lived, albeit in another galaxy, as Lilandra had to depart with her lover to seek medical attention, Magneto kept his promise as headmaster of Xavier’s School for Gifted Youngsters. Taking the alias of Michael Xavier, Charles’ purported cousin, Magneto took charge of the New Mutants - the next generation of Xavier’s dream.
Magneto’s tenure as the headmaster was relatively short, with many circumstances beyond his control conspiring against him. Eventually, the X-Men were believed dead and the New Mutants disappeared on one adventure after another. The final nail in the coffin of Magneto’s redemption was the destruction of the school itself by a peripheral conflict called Inferno. With no school, much less students to attend, Magneto left - slowly returning to his old ways. This process came full circle with the arrival of the Acolytes, who managed to convince Magneto once again that mutant-human cooperation was impossible - and the only way for survival was through strength.
Over the years, there were many opportunities for Xavier and Magneto to both end the life of their counterpart with swift action. However, it seems that both longed for a more decisive victory, won on the battlefield of ideas and ideology. All of that was abandoned by Xavier, however, when he witnessed Magneto’s horrific attack on Wolverine.
When the specter of Magneto again shadowed over the globe, the X-Men once more rose to stop him, this time traveling to the would-be despot’s orbital headquarters, Avalon. Wishing to end the cycle of conflict then and there, Wolverine attacked Magneto, slicing him with his claws. In retaliation, Magneto used his powers to remove the adamantium alloy that had been bonded to the X-Man’s skeletal structure - a most painful process. [X-Men (2nd series) #25]
Incensed by this action, Xavier could stomach no more, and used his telepathy to remove Magneto’s greatest weapons - his hatred, his ego… his nightmares. The procedure left Xavier exhausted, but it left Magneto in a vegetative state, incapable of any sign of intelligence or action. He was left in a waking coma.
With Wolverine in critical condition, the X-Men returned to Earth, leaving Magneto in the care of Colossus and the other Acolytes. In time, Avalon was destroyed and Magneto was believed dead once again. As before, he recovered and re-emerged, as deadly as ever. In the meantime, however, his hatred, his ego and his nightmares lived on… within Charles Xavier.
After many months, it was discovered that the enigmatic new foe that had been bedeviling the X-Men - known only as Onslaught - was in fact Xavier. This mystery of why this man of peace and tolerance could be so transformed was soon revealed - Xavier had not wiped the rage and fury of Magneto… he had inadvertently absorbed them. The maniacal nature of Magneto, added to the psyche of the world’s most powerful telepath had led to a conflict which threatened to engulf the entire island of Manhattan.
Eventually, the X-Men, as well as numerous other heroes, such as the Avengers and Fantastic Four, were able to return Xavier to sanity. However, no one would ever look at Xavier in the same way again - including Xavier himself. His brief tenure down Magneto’s philosophy had been terrifying - and deadly.