1. The science of disease is incredibly difficult, and it could just be that the drug industry is going through a tough phase right now—possibly because companies have been focusing on "complex and chronic diseases such as cancer" that have higher failure rates. Plus, they haven't quite mastered all the new-fangled technology—such as genomics—but once they do, innovation will really take off.Anyway, that's the rough overview. Obviously there are a lot of things that could be fiddled with—one of GAO's recommendations was to create a multi-tier patent system, in which, say, a drug company could receive, say, a 30-year patent for a drug considered highly innovative, but would get a much shorter patents for "me too" drugs that don't much improve on existing drugs. As an alternative, Dean Baker has proposed a publicly-funded alternative to the patent system that could work just as well, or better.
2. There's an industry-wide lack of trained researchers "who possess the ability to effectively translate basic discoveries into new drugs." In particular, the number of physician-scientists—people who have both medical and research degrees and can bring the two realms together—declined by 22 percent from 1983 to 1998. Some blame academic institutions for not providing enough financial incentives to convince medical students to pursue these tracks.
3. Over the past ten years, drug companies have increasingly concentrated on producing "blockbuster drugs"—that is, drugs intended for large populations that have the potential to reach over $1 billion in annual sales. This sort of work can crowd out research on other smaller, potentially innovative drugs that would benefit more limited portions of the population, or be used to treat less-visible diseases.
(Moreover, since multiple drug companies are often pursuing the same blockbuster drug for the same blockbuster market, one company will often beat the others to the punch, and the losing companies will have to discontinue all the research they've done.)
4. Given that drug development has become more costly and more complex of late, many companies prefer to focus on "me too" drugs that are basically variations on drugs already on the market. These drugs are less innovative, but they're also less risky to develop.
5. As a follow-up, the current intellectual property system encourages "me too" drugs, since companies can obtain new patents for their drugs regardless of whether the drug in question is a significant improvement on what's already out there. Instead, they can slightly tweak an existing drug and voila—they get a patent and the profit that comes with it.
6. According to industry analysts, the recent rash of mergers and acquisitions in the drug industry might have negatively affected drug development. Quite often, a newly-merged company will simply cancel several of the former company's ongoing projects because there are new revenue expectations to meet. In one 2003 survey, nearly half of all clinical research organizations reported that they had had drug development projects cancelled due to mergers.
7. Many of the FDA's rules concerning drug safety and effectiveness are unclear, and that uncertainty may sometimes lead companies to abandon drugs in the early stages rather than risk bringing a drug all the way to the final stage only to get rejected by the FDA. The fuzzy rules, though, are partly due to an honest lack of consensus about how to measure drug safety. Other industry officials have also complained that the FDA takes too long to review a drug, although most of these rules were put in place for safety purposes. (I'd note, though, that the FDA is, to a large extent, in the pocket of the drug industry, and hardly the most adversarial force around.)