The first suggestion that Deadpool was actually mentally unstable came when he was captured by the Gamesmaster and Dr. Weisman at their asylum. [X-Force (1st series) #47] Further elaboration came in Deadpool's origin story a few years later, explicitly stating that his time in the Workshop broke him mentally, beginning with his suicidal depression and leading to uncontrollable mania once his healing factor kicked in. However, a specific diagnosis never arose. [Deadpool/Death Annual '98]
During Joe Kelly's run on Deadpool (2nd series) #1-33, Wade Wilson was portrayed as a deeply troubled individual, for sure. He had a cynical view of people and the world, identifying them as inherently flawed and even malicious. His various attempts to become more heroic for Siryn or LL&L clashed repeatedly with this worldview, leaving him mentally off-balance. He was highly dependent on the positive reinforcement of others in order to maintain his self-esteem. Wilson demonstrated psychotic and sadistic tendencies, not only in combat situations but in his toxic domestic relationship with Blind Al. It was also during this run and his previous miniseries where Wade showed a crippling phobia of letting his scarred face be seen by others. At times, he even referred to his Deadpool mask as his true "face." In short, he was a functional human being for the most part, but suffering deep antisocial tendencies.
Deadpool actually received psychological counseling for the first time in this series, although it was mostly in response to a specific breakdown he suffered after killing the alien messiah, supposedly for the greater good. Deadpool was experiencing hallucinations that interfered with his ability to process reality. [Deadpool (2nd series) #26-27] This also led to a deeper revelation that Deadpool was suppressing memories, including apparently his entire married life from before becoming a mercenary. Due to the later revelations about this scenario, however, it's hard to say if Wade was truly suppressing memories here, or if they never existed in the first place. [Deadpool (2nd series) #32-33]
It would be some time before the instability of Deadpool's mind was explicitly linked to the cellular regeneration his body was constantly undergoing. Early stories made passing reference to this, such as Taskmaster being unable to predict Deadpool's moves or Vamp finding his mind remarkably untidy with her telepathy. However, it was not declared to be superhumanly unusual at this time. [Deadpool (2nd series) #2, 6] Later on, however, the benefits and detriments of his unique brain were specifically linked to his healing factor. Deadpool's constantly regenerating cells made his thoughts "slippery," giving him an immunity to telepathic probes and manipulation of various kinds. However, it also made it increasingly difficult for Deadpool to hold on to his own thoughts. He demonstrated short- and long-term amnesia problems as well as symptoms similar to Attention Deficit-Hyperactivity Disorder. [Cable & Deadpool #8-14] The more extreme issues with his mind were eventually cleared up in a deep immersive effort by Cable's telepathic and telekinetic powers. [Cable & Deadpool #18]
Absorbing the body and mind of the insane Madcap caused Deadpool severe issues as well. During the time that Madcap co-occupied Deadpool's body, he began experiencing acute schizophrenia. Deadpool would hear voices in his head, representing his own mind and Madcap's separate but integrated mind. Although he recognized the second voice only appeared after his encounter with Madcap, Wade appeared unaware that the second voice in his head actually WAS Madcap. Deadpool was highly susceptible to influence by this voice, suffering changes in his memories and behavior. Wade also became subject to vivid visual hallucinations, which he nicknamed "Pool-O-Vision."
An extension of Deadpool's insanity (of a sort) was his ability to recognize that he was a comic book character. There were some early signs of this trait. In his first ongoing series, Deadpool was caught in the jungle narrating his own adventures, in what was presented to the audience as traditional narration boxes. [Deadpool (2nd series) #1] On another occasion, he remarked about a flashback that was just wrapping up in the comic. [Deadpool Team-Up oneshot] These were not conclusive events, though... Deadpool was referring to common literary tropes like first-person narration and flashbacks, but he didn't specifically say he was doing so because he WAS a fictional character.
Still, it became more common for Deadpool to make remarks like this. A number of references to page length and raising sales were made in a Deadpool Wizard Special. [Deadpool (2nd series) #0] He also had an encounter with Bullseye where he specifically mentioned their last meeting was in "issue sixteen." The tendency wasn't entirely unique to Deadpool at this point, though -- Shadowcat lamented about having to appear in "a non-X-Book" during the same set of issues. [Deadpool (2nd series) #27-28]
Therefore, it has to be Christopher Priest's run of Deadpool (2nd series) #34-45 that was most famous at the time for cementing Deadpool's view that he was a comic character. In almost every issue of the run, Wade made commentary about how none of his circumstances were real, beginning to explain "You see, there's a man with a typewriter..." before repeatedly getting cut off. The entire run was book-ended by Deadpool visiting Limbo, where the various heroes of Priest's past cancelled titles tried to make him one of them, only for Wade to push on to maintain sales and even kidnap and deposit Priest into Limbo instead, letting his title continue under a different writer.
Other confusing or contradictory aspects of Deadpool's character have emerged over time as well. Deadpool's place of birth has fluctuated between America and Canada. In his second appearance, Wade expressed that he had always hated "this country," referring to Canada. Other stories emphasized that Deadpool was a product of a Canadian super-soldier program, but not necessary whether he himself was Canadian. In Cable & Deadpool #19, Deadpool supposedly regained his true memories of being born in Ohio. He continued to identify as American throughout that series, considering the President "his" commander-in-chief, and so on. However, Deadpool began calling himself Canadian again after that series ended, and Deadpool (4th series) #19 seemed to confirm the Ohio story was one of Butler's memory implants.
Deadpool also has a habit of claiming to be a mutant. His healing factor was identified as a gift from the Weapon X Program as early as Deadpool: The Circle Chase #3, but it wouldn't be until Deadpool (2nd series) #1's recap pages that it was explicitly stated to be derived from Wolverine's genes. Deadpool would refer to his "mutant healing factor" in Wolverine (2nd series) #88, which might technically be true since the healing factor came from a mutant. Starting with Cable & Deadpool #15, though, Deadpool would occasionally refer to himself as both a mutant and an X-Man, typically prompting a correction from any real mutant or X-Man in earshot.