Amazingly enough, other IDS companies have open signature sets and do not have any those predicted issues. What closing the signature set DOES do, to the analyst using the system, includes:
- Blinding the analyst to what traffic actually triggered the alert, making it difficult at best to determine if the alert was a false or true positive.
- Prevents the analyst from modifying the signature to narrow the scope to prevent too many false positives, or using it as the template for a new signature customized the company's individual needs.
- It keeps the analyst from sanity checking new signatures, to see if the vendor is putting out intelligent, well researched signatures, or just throwing out a very generic signature that will also trigger on hundreds of other packets with similar packet data. Companies will do this to allow them to say their signature base detects threat xyz, regardless of whether it's a well written signature or not.
Another thing to ask, if the signature set is open, is whether or not the signatures are written in a proprietary language or not. Being able to write signature by just following the prescribed format (like Snort or Dragon) is a great boon to the analyst, and cuts way down on the learning curve to get operational. Anyone can learn to write a Snort signature in a short time. However, learning to write a good Snort signature takes a good deal more investment of time..