If going by numbers in America than yea but it's because the white population is at 63% while us blacks are only 13%. That's what sociologist mean when they say institutionalized racism is mostly from whites. White privilege does not mean every white person has it easy and that every non white has it hard. It doesn't mean all white people are racists out to hold minorities down. It just means being part of the majority group is usually more beneficial than being a minority in most countries. The white privilege people refer to is based on strictly American population and race demographics. There are always exceptions. I probably had more privilege than some whites since my black parents were/are upper middle class architects. There are poor and disadvantaged white people but it doesn't negate an over all trend of white privilege.

Also overt personal racism is different than institutionalized racism which is on a macro systematic schale. Any race is capable of racism. I have seen the white and asians bullied when I went to a majority black school. A group needs to dominate an area or be part of the majority to be part of large scale systematic institutionalized racism. This doesn't mean there is group of white men maliciously trying to bar minorities from jobs. Are there white people like that? Yes but they are probably much smaller than the ones who do it unconsciously. The concept of homogeneity dictates that we are more likely to mate with, be friends with, date and give unconscious favoritism to those similar to us. This could be in race, socioeconomic status or hobby. It's a big part of the reason white people usually date other white people. It's not because whites are racist or hate the look of other races. This phenomenon carries over to the job market. Remember whites are 63%(72 if you count white hispanics like those from spain). This is what affirmative action tries to correct for but it's not a perfect law and sometimes hurts white people.