This is happening partly because of the often low standards of proprietary software. Companies have all been through many phases when they have introduced new proprietary software packages. These transitions often do not go well, end up going way over-budget and over-time. If the software is open source, it is often freely licensed, saving the company the licensing cost. Even if the introduction of the software goes over-time, it at least is unlikely to go so over-budget.
There is usually little or no review of proprietary closed-source security tools. This means that the vendors can base a large part of their security on the basis that no-one has access to their source code, so they can implement "security by obscurity". Furthermore, as peer source-code review cannot happen, bugs are not usually discovered for a long time, if ever. However, the "black hat" community will devote significant resources to breaking the security of the systems, giving them a major advantage. There are several companies, such as eEye Security, who have made a very healthy profit through attacking closed-source code.
On the other hand, open-source developers know that their source code will be examined carefully by potential attackers, and must therefore work much harder to protect against attacks. They cannot rely for one minute on "security by obscurity" as it simply doesn't exist in the open-source security world. Features must be carefully thought out, well designed and well implemented to avoid security holes. The system is designed and implemented on the assumption that the system will be attacked by people who have a full understanding of how all parts of the system work.
Building a new door-lock in a world where everyone is a locksmith or a burglar is much harder than building one in a world where people cannot see the innards of the lock. As a result, the door-lock produced in the former situation is much stronger than the latter.

Usually because of valid business concerns, such as:
Will the product development continue? How can they purchase maintenance and support contracts? Will they actually be able to get help when they need it?
If a software package is only maintained by 1 person, what happens if they decide not to continue development any longer? The answer to this is that virtually all open-source applications are known and understood by a team of people from around the world, so the disappearance of 1 person has no long-lasting effect on the development.
Maintenance and support contracts are areas in which the open-source community have traditionally been somewhat lacking. However, the community knows this is a very valid concern, and so the business world has stepped in to fill the gap. Companies such as Fortress Systems Limited and LinuxIT were formed to provide commercial-grade support of open-source community software packages and systems. They are geared up to provide SLA's and liaise with the development team as necessary when resolution of a customer's problem is beyond their knowledge of the product. As the relationship between the developers and the companies progresses, the companies providing the support will learn how to resolve virtually all problems themselves. But they will still be able to directly contact the original developers directly, a position which is often difficult or impossible in closed-source systems.
Resellers of closed-source systems rarely have direct email access to the original developers of the product, instead having to go via several layers of support staff at the original vendor's company. They are therefore limited in what help they can get. In the open-source world, the support companies can always directly contact the development team, gaining support from the authors themselves. Customers are always very impressed by the speed and quality of support available from developers of open-source systems. This is not simply because the software is open-source, but is because of the development community model in which open-source products are normally developed.

The most common problem is the actual installation of the open source software package. Proprietary packages, however poor their performance and functionality might be, usually spend a large amount of time and effort creating the installer using a closed-source installation package. The costs of the most common installation systems are beyond the funds available to the open-source community, and so installation is often slightly more awkward, relying more heavily on the knowledge and skills of the engineers installing the software.
Most open-source security systems do not run on Microsoft Windows, due in part to the huge learning curve required before a decent high-performance package can be written. Due to its nature as a tool box of interacting, but separate, components, it is usually far easier to write security applications for Unix or Linux based systems. Microsoft Windows is very much one integrated system, where you are restricted in your actions by what Microsoft chose to let you do in separate parts of the system. This isn't to say that these applications cannot be written for a Microsoft platform, it is that their design can be far more modular and insular in Unix-based systems, without the possibility of causing any awkward reactions in other unrelated parts of the system.
This results in problems for companies that only know how to run Microsoft Windows based systems, as suddenly they are going to need to be able to run a Unix or Linux system for a new application. This is not hard, but it is new to them. The system administrators may need to attend training courses in Unix, as well as training for the security application if necessary.

Almost all the closed-source commercial security vendors spread FUD against open source. The most common thing they say is that their systems are better and that the TCO is lower for their systems. They have far larger marketing budgets than most of the open-source community, and are quite happy to portray very minor features and major new "technologies". The open-source community is usually more honest, and will only push features and techniques that make a real example. That may make them appear to be more naive, from a closed-source commercial point of view. One classic example is Qualcomm's advertising of Eudora's new "Launch Protect" technology to protect users from opening dangerous attachments. It actually consists of one dialog box which is presented when the user clicks on an attachment. All this does is make the user click one extra button to view the attachment. It doesn't add any real protection at all, but Qualcomm pushed it as a major new feature.
The TCO concern is pushed on the basis that the closed-source vendors have these wonderful teams of technical support staff who will solve complex problems in an instant. Anyone who has dealt with most of these companies will be painfully aware of the real truth of this marketing. How many closed-source vendors will give you the email addresses of individual members of the development team?

They are very wrong when it comes to the quality and speed of support available. Current users know how good the support for open-source applications can be, compared to the paid-for support they are getting from closed-source vendors. If you want to see an example of this, take a look at the MailScanner "user testimonials guestbook" where users write their honest opinions of the value of the software itself as well as the quality and speed of support they get.
The guest book contents is completely unedited except for the removal of advertising spam.

Very little. The main problem is being able to purchase commercial support contracts. Most organisations are happier knowing that there is always someone they can contact by phone or email, regardless of the time of day or the fact they are paying for this support. More companies like Fortress Systems Ltd and LinuxIT are needed to provide commercial support.