The history of the wizarding world began in prehistoric times, when a benevolent mutation spread among humanity. This mutation consisted of certain genetic traits that enabled people to mentally access and control certain energies and entities from universes of non-Einsteinian physics that are congruent to our own. Because these other universes possess physical laws that are tremendously different from our universe, those people were able to defy the conventional laws of physics and alter many aspects of reality on a fundamental level by using these energies and entities to manipulate both the space-time continuum and the fundamental interactions of nature. Simply put, they could use powers from other universes to reshape ours on intrinsic levels. Although compared to the scope of the entire universe, it was within a relatively microscopic and localized area. This is the source of the phenomenon that is called magic.
The existence of the Neanderthals is the evolutionary pressure that caused the magic-using trait to evolve in Homo sapiens. Humans won out and the Neanderthals died out when modern man appeared on the scene because many humans had magic, while not a single Neanderthal possessed that gift.
With no spells or wands, these primitive magic wielders manifested their abilities rather subtly, it appeared as frequent cases of good luck or happy accidents. However, some of these people acquired some rudimentary control over their power--enough to aid in the hunt and in causing or curing various simple maladies. With this advantage; Homo sapiens was able to supersede the Neanderthal and flourish; driving the less fortunate primitives out.
For a long time, humans of magical stock had a significant place within most societies. The data for this can be found in our ancient texts; including the Bible, Qur'an, Talmud, Dead Sea Scrolls, Book of the Dead, etc. During this time, some magic wielders used magic to perform biological experiments on both humans and animals that permanently altered the DNA of these creatures. These experiments led to the existence of elves, giants, goblins, centaurs, trolls, dragons and all the other magical beings and beasts of the world.
It is because of social pressure that the magic-using trait, which is so vital to our past, receded to the small population we see today. An actively anti-thaumaturgical mindset spread across the world, into various cultures for various reasons. People who manifested magical abilities were hunted down and eliminated. Only people who had somehow learned to suppress their abilities survived (magic manifest itself around the age of puberty, if training is withheld during this critical period the ability can never thereafter be fully developed or controlled).
Eventually it was discovered that wood is a magically imbued material. This is because in addition to harnessing sunlight, the photosynthetic process also has a resonating quantum effect that slightly augments the special protein structure that witches and wizards possess. Using the unique properties of wood, the first magic wands were created. Wands function as a "magic focuser". In the same way a convex glass can focus sunlight enough to create a small fire, a wand focuses the magical potential of its user, not necessarily increasing the magic power, just making it more intense.
The development of the magic wand allowed for the first time the controlled and reliable applications of magic. This was as much a disaster for the thaumaturgically inclined as the anti-magical factions of humanity - now the magically inclined had a clear and decided advantage over the non-magical or "Muggles" as they came to be known to the "wizarding world", and these Muggles now had a powerful political and economic incentive to hunt down and eliminate the magical. Eventually those with the magic gene complex declined in number.
Shortly after the discovery of the wand, many magical people began exiling themselves from the Muggle world, forming small and isolated magical communities. With the increasing reliance among Muggles on scientific reasoning, the break between the wizarding and Muggle worlds became more and more complete. Each culture went on to create their own civilization: social structures, economies, governments, etc. Each borrowed a little from the other as the years went by but it became apparent that the Muggles must be kept ignorant of the existence of their magical kin for their own good.
In Europe, during the Middle Ages magic was seen as 'heresy' and violently persecuted. Most common magicals were either hunted down and destroyed or suppressed their ability to hide their true nature. As a result, the trait atrophied. Only those that kept their magic active and stayed alive by secluding themselves kept the ability strong. Thus, the magicals collectively decided to form a separate society, using their craft to hide their nature and locations from the increasingly large Muggle populations. With the passing of the International Statute of Wizarding Secrecy in 1692, the entire wizarding world formally severed contact with mundane society, and removed itself from it, retiring behind a "firewall" of illusions and Muggle-repelling charms. The only Muggles to know of this secret, magical world were those Muggle husbands, wives, parents, siblings and offspring who chose to accompany the wizards into their sanctuary. The ruling council of the wizarding world, the Ministry of Magic, took steps to limit the likelihood of wizards and witches encountering Muggles either by accident or design. This not only completed the separation of the two cultures but also laid the responsibility on the various wizarding governments in each country for maintaining the secrecy of everything from Quidditch games to dragons. Over the years, for Muggles, magic became the stuff of fairy tales and legend.
While the articles of the International Statute of Wizarding Secrecy allowed for the recruitment of Muggle-born magical children, once "Seclusion" was established the actual incidence of such recruitments were exceedingly rare because under such conditions it was virtually impossible for any Muggle-born magical child outside the boundaries to have been noted or identified.
About a century after the formal adoption of Seclusion, it became apparent to those who were in a position to oversee the wizarding world's welfare, that its population's birthrate was insufficient to maintain its numbers at a high enough level either to survive over the long term. In addition, a discernible rise in the number of magical children being born outside the wizarding world was reaching a point that they were becoming a security risk. The belief among Muggles that magic does not exist could only be fostered if there were no wizards practicing magic out in mundane society and unintentional or not, these children were practicing magic.
It was ultimately decided among the wizarding world's leaders to make a virtue of necessity and to take aggressive steps to identify, train and assimilate these Muggle-born magical children into the wizarding world, both from a security standpoint by removing them from the awareness of Muggles, as well as to take possession of these children's talents to offset natural attrition.
Early in the first half of the 19th century, a cross-section of the wizarding world, who had nothing but bad associations with Muggles - dating from the era that Seclusion had been determined to be necessary for wizards' continued survival - began to put a renewed emphasis upon the need to distinguish those who were descended from "pureblood" wizarding ancestry, from those who were "halfblood" or Muggle-born. The wizarding world has been dealing with the ramifications of this situation and the response to it ever since, with varying degrees of success.
So the evolution of magic in humanity underwent three distinct phases: prehistoric--magical abilities were selected for as they improved the luck of the individual, suppression--uncontrollable magical manifestations were selected against as they violated the mores of the societies, isolation--people with magical abilities formed their own societies; strong magical abilities again became desirable.
The trait of the magic using ability is purely a genetic difference and there are a variety of genetic factors controlling the it. There is no single gene for the ability, instead it is scattered throughout the genetic material and the combination determines magic using strength. Generally, the children of magicals are also magicals; although, this is not always the case. For magicals the production of a Muggle child is extremely unlikely; while for Muggles the production of a magical child is even more unlikely.
There is a spectrum of magical abilities in humans, ranging from no discernible magical ability even with training, through magical abilities only manifested with training, up to spontaneous magical manifestations. The majority of humanity falls into the middle category (although due to various selection processes the median of abilities in the Muggle world is well to the left of the median of abilities in the wizarding world). The offspring of magical parents who fall into either of the latter two categories are trained into magic-using people. Those few children in the Muggle society that fall into the third category can be brought into the magical realm, as their magical powers are well above the Muggle norm.
The current population of the wizarding world stands at approximately 25% Muggle-born witches and wizards, 50% halfbloods, and 25% purebloods. Traditionally, all sixteen of one's great-great-grandparents must have been magicals to be considered a pureblood: some more severe schools of thought require several more generations of magical lineage to be considered purebloods. There are no 'true' purebloods left; many of those who call themselves such simply strike Muggles and halfbloods from their family records.
Almost all magic is done with the use of a supporting tool, typically a wand. By using a wand one's magical powers are greatly focused. Furthermore, most actual spell casting is done by using short incantations (most often in what sounds like a modified form of Latin) accompanied by gestures. This is because certain words and gestures act as psychological triggers to alter the perceptions of the mind, allowing one to summon an extradimensional force to override the usual laws of nature. Potions and talismans also serve as psychological conduits to summon a power. Accomplished wizards and witches can sometimes perform magic, especially simple magic, without the need for any sort of conduit.
Regardless of how powerful a witch or wizard is, he or she is by no means without limits. For instance, while it is possible to conjure things out of thin air it's far more tricky to create something that fits an exact specification rather than a general one - moreover any objects so conjured tend not to last.
Most magic is relatively neutral--it can be used for bad or good. Some magic, however, is evil in it's intention through and through. Spells of this kind are often called curses. Curses are spells that are often intended to cause harm to another person. This intention to do harm places that spell into the realm of the Dark Arts. However, simply casting a Curse spell doesn't mean that a person is using the Dark Arts. Ultimately, the deep, true intention of the caster is what makes the difference. Dark Magic is more than simply curses, however. Magic which involves tampering with the free will of another person or which kills another person would be considered Dark Magic.
Some aspects of the wizarding world appear less than modern in character, even old-fashioned and quaint; others are more advanced than Muggle society. The technological development of the wizarding world is ostensibly behind Muggle technological development - though, in fact, a large number of technologically-complex devices do exist, and the use of magic renders Muggle convenience technologies unnecessary. In addition, magic has a tendency to cause interference with electrical equipment and Muggle devices (such as cameras and radios) are able to power themselves on ambient summoned magical forces. Such examples are rare, however; wizards rarely make use of Muggle technologies nor do they have much interest in doing so.
The wizarding world not only exists alongside the Muggle world, but is embedded within it. The vast majority of wizarding world locations are isolated within the wider non-magical area. These locations are hidden by a combination of magical protections (many magical locations, such as the island of Drear off the coast of Scotland, and the wizarding prison, Azkaban Fortress, are rendered unplottable, or impossible to locate on a map) and the natural tendency of everyday, non-magical people to ignore anything they cannot explain or understand. Although wizarding society lives for the most part directly amongst Muggles, few wizards and witches are aware of basic Muggle culture.