Here's the scoop from someone in the field:
There's tremendous pressure in academics to be famous, or bring in lots of money in the form of grants, or both. Some of this is internal pressures (like achievement motivation), but some of this external pressures (keeping your job or salary). This leads to a focus on publishing attention-grabbing results over "truthfulness" for lack of a better way of putting it, and getting published even if the results aren't actually worth publishing.
Some of the things that have caused this to spiral out of control:
Universities encourage this because of what are known as "indirect costs" that are charged on grants. If you're a researcher, when you apply for a grant, you don't just ask for money to run your study, you ask for money to give to the university, ostensibly to pay for the cost of you just being there (e.g., heating your lab space, maintaining the building, etc.). Although this is sort of reasonable in theory, in practice, the money taken in by the university far exceeds the actual costs to them by the person doing the research (not to mention the fact that they are supposed to be supporting this themselves to some extent). So the research is seen as a profit generator in a literal sense, which leads to pressure to get grants regardless of their scientific merit.
Journals want to publish flashy, attention-grabbing results because it means more people will read them. So they sort of turn a blind eye at some level to certain issues. In theory, they should "notice their data is skewed and not publish them," but they're willing to overlook what's easy to overlook if it means more citations to their papers. I've seen some journals literally tell reviewers not to pay attention to the quality of the research design, only the attention-grabbing-ness of the papers, like "hey, we'll worry about that--you just tell us if it's faddish enough."
Faddishness is another problem. No one really knows what the right direction to go in is, so there are tons of fads, which are made worse by social dynamics. Research has actually shown that the biggest predictor of getting a grant is being socially connected to people on the grant review panels, and people on the grant review panels tend to be those who have had grants... you see where this is going. People lots of times evaluate research on whether it supports their viewpoint, or they like what it says, because they don't really know what else to do, because people don't really know what the right answer is.
Research results are random. That's why you have statistics. So you can keep trying over and over again, or tweaking things just so, until you get the story you want. Even if you're not doing it intentionally, you can wind up with it by chance, which just then reinforces what you're doing. This is reinforced by the focus on single study results rather than a body of literature.
Universities are increasingly under pressure to be run as a business, and under fiscal scrutiny, so they're replacing tenured professors with untenured research staff. The net result of this, though, is to amplify the pressure to produce flashy results, and bring in more money. Undermining tenure means money and fame becomes more of a driving factor in scientific research, because the researchers' salary depends on it.
Training has sort of become a pyramid scheme. Doctoral students are brought in to function as workers, not as students. This increases the numbers of persons competing for positions after graduation, and makes everything worse as a result. If one of the graduates is lucky enough to get a job, they are under pressure to do the same thing, which then perpetuates things further.
I could also mention the nature of credit, where we tend to want to attribute things to single individuals when it's more of a team effort. This leads to all sorts of problems where people deify and idolize certain researchers, and lots of taking credit for others' work, or not giving credit to people who deserve it (this is just as true of the consumers of the research as it is the producers).
Finally, in a lot of ways, overhead bs has gotten worse, in the form of increased federal regulation, ethical review boards that nitpick over useless details of no real ethical consequence whatsoever, etc. This increased bs increases the costs of the research, and creates greater incentive to get something out of what you've put effort into.
Also, I'm just talking about university research--industry involvement introduces a whole other set of problems.
It's a total mess.