Data Point
One of the first aspects to be considered is the importance of the Data Point. For product developers, it holds immense importance since it allows them to understand the main issues. Similarly, from the point of view of the system admins, it’s a great option to get a prior warning of a new threat and ensure that the system is well prepared for an onslaught.
Data Sources
There is no surprise in the fact that most IT companies are putting in tremendous amounts of efforts to gather all kinds of data.Most products nowadays include a feature known as the phone home which reports back to the base station as soon as a thread is spotted. Most products also use cloud lookup systems that help in recording innumerable data pertaining to what is being looked up by the servers.
Limitations in detection of data
The biggest problem is in the detection of data. Since it is extremely hard to report things that cannot be seen, a lot of important data go unreported until the time that they are detected and detection techniques are implemented on them. It is an extremely problematic issue for most testers especially when they try to test protection methods against the zero day threats.
Multiple Sources
The limited versions can be vastly improved by increasing the range of sources from which data is collected. A cross industry IEEE initiative has been already made available and offers a standard format to facilitate the sharing of metadata. It also operates along with the existing systems which are based on sample sharing. Once implemented, the system hopes to offer opportunities for a simplified and much more accurate merging of data.
Clustering
Clustering is another issue which has been around for quite some time. One of the main problems has been to clearly define what a threat truly constitutes of. Mostly, the unique items are recorded by file hashes when binary files are being looked up. However, most of the times, a single piece of malware is morphed innumerable times, either locally or at the server side. Thus, a single file hash might not succeed in detecting them more than once and fails to classify them as a much bigger threat. This issue can be avoided by simply splitting the threat data with the help of detection ID’s rather than using file hashes. Since the detection ID’s continually try to cross match reports amongst different products, they will rarely match which makes accurate clustering quite difficult. The same is true for URL related prevalence information as well.
It is quite clear that telemetry data and the issues of prevalence are extremely vital and even more difficult and tricky to handle or gather at the present moment. However, the innovations in the field do hold numerous promises. The emergence of cross industry and cross sector collaboration has also give hope that the problems of producing usable and reliable insights on what is actually going on can truly be overcome.
You can follow any responses to this entry through the RSS 2.0 feed.
Leave your comment